JP2006319944A - Decoding control device and method, recording medium, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable to more rapidly decode a video image stream. <P>SOLUTION: A decoding region setting section 563 sets divided decoding regions to be decoded by each of decoders 565 by dividing the slices in a picture in order from the top for each of the number of divided slices obtained by dividing the number of slices in a picture of a video image stream by the number of decoders 565. A decoding control section 564 controls each of the decoders 565 so as to execute variable length decoding and dequantization of slices in the divided decoding region assigned to each of the decoders 565 in parallel to the other decoders 565, and controls a decoding section 601 so as to execute inverse discrete cosine transform in parallel to decoding by each of the decoders 565. This device is applicable to a video reproducing device for reproducing a video image stream. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a decoding control apparatus and method, a recording medium, and a program, and more particularly, to a decoding control apparatus and method, a recording medium, and a program that realize high-speed decoding of a video stream.

  Conventionally, in order to quickly start decoding a video stream, the recording position of a frame specified by the user is recorded as index data at the time of encoding, and the position to start decoding is detected using the index data at the time of decoding. Has been proposed (see, for example, Patent Document 1).

  In order to speed up the decoding of the video stream, it is conceivable to divide the decoding process into a plurality of threads and perform the decoding in parallel with a plurality of processors. For example, in the case of a video stream encoded by the MPEG (Moving Picture Experts Group) 2 method, the decoding process is divided with a slice in a picture as one unit. Specifically, when the decoding process is divided by four processors, as shown on the left side of FIG. 1, when 16 slices are included in one picture, the top slice in the picture is ordered. Each processor decodes one by one in parallel. That is, each processor has slices 1-1 to 1-4, slices 2-1 to 2-4, slices 3-1 to 3-4, and slices 4-1 to 4- shown on the right side of FIG. 4. Decode any of the four sets of four slices.

JP 11-341437 A

  However, when the processing is divided as shown in FIG. 1, each processor needs to detect the position of the slice to be decoded each time one slice is decoded. The time required increases. Furthermore, even when the invention described in Patent Document 1 is applied, the time required for detecting the decoding position of each processor is hardly reduced because the time for detecting the slice position is not shortened.

  The present invention has been made in view of such a situation, and makes it possible to decode a video stream at a higher speed.

  The program according to the present invention is a divided area obtained by dividing an area corresponding to a picture of a video stream according to the number of first decoding means, and the first decoding means and the second decoding for decoding the video stream. A setting step for setting a divided area including a plurality of unit areas which are processing units of the means, and decoding up to a predetermined middle stage of the unit area in the divided area respectively assigned to the first decoding means The first half decoding control step for controlling the first decoding means to be executed in parallel with the first decoding means, and decoding to the middle stage in parallel with the decoding by the first decoding means A computer is caused to execute processing including a second half decoding control step of controlling the second decoding means so as to execute decoding of the remaining stages of the unit area.

  The video stream may be a video stream encoded by a moving picture experts group (MPEG) method.

  In the first half decoding control step, the first decoding means is controlled to perform decoding including variable length decoding and inverse quantization of the slice, and in the second half decoding control step, the inverse discrete cosine transform of the slice is performed. The second decoding means can be controlled to perform decoding including

  The unit area may be a slice.

  The first decoding unit and the second decoding unit may be realized by different hardware.

  The second decoding means can be realized by a GPU (Graphics Processing Unit).

  In the latter half decoding control step, information indicating how far the unit area has been decoded is supplied to the second decoding means.

  The recording medium of the present invention records the program of the present invention.

  The decoding control method of the present invention is a divided area obtained by dividing an area corresponding to a picture of a video stream according to the number of first decoding means, wherein the first decoding means and the second decoding means for decoding the video stream A setting step of setting a divided area including a plurality of unit areas which are processing units of the decoding means, and decoding up to a predetermined middle stage of the unit area in the divided area respectively assigned to the first decoding means The first half decoding control step for controlling the first decoding means so as to be executed in parallel with the other first decoding means, and decoding up to the intermediate stage in parallel with the decoding by the first decoding means And a second half decoding control step of controlling the second decoding means so as to execute the decoding of the remaining stage of the unit area.

  The decoding control apparatus of the present invention is a divided area obtained by dividing an area corresponding to a picture of a video stream according to the number of first decoding means, wherein the first decoding means and the second decoding means for decoding the video stream Setting means for setting a divided area including a plurality of unit areas which are processing units of the decoding means, and decoding up to a predetermined intermediate stage of the unit area in the divided area respectively assigned to the first decoding means The unit region is controlled to be executed in parallel with the other first decoding means, and is decoded to the intermediate stage in parallel with the decoding by the first decoding means. Decoding control means for controlling the second decoding means so as to execute the remaining stages of decoding.

  In the program of the present invention, the program recorded on the recording medium, the decoding control method, and the decoding control apparatus, the area corresponding to the picture of the video stream is divided into areas divided according to the number of first decoding means. The divided areas including a plurality of unit areas, which are processing units of the first decoding means and the second decoding means for decoding the video stream, are set and assigned to the first decoding means, respectively. The first decoding unit is controlled so that decoding up to a predetermined intermediate stage of the unit area in the area is performed in parallel with the other first decoding unit, and decoding by the first decoding unit is performed. The second decoding means is controlled so as to execute the decoding of the remaining stage of the unit area decoded up to the middle stage in parallel.

  According to the present invention, a video stream can be decoded at a higher speed.

  Embodiments of the present invention will be described below. Correspondences between constituent elements described in the claims and specific examples in the embodiments of the present invention are exemplified as follows. This description is to confirm that specific examples supporting the invention described in the claims are described in the embodiments of the invention. Therefore, even though there are specific examples that are described in the embodiment of the invention but are not described here as corresponding to the configuration requirements, the specific examples are not included in the configuration. It does not mean that it does not correspond to a requirement. On the contrary, even if a specific example is described here as corresponding to a configuration requirement, this means that the specific example does not correspond to a configuration requirement other than the configuration requirement. not.

  Further, this description does not mean that all the inventions corresponding to the specific examples described in the embodiments of the invention are described in the claims. In other words, this description is an invention corresponding to the specific example described in the embodiment of the invention, and the existence of an invention not described in the claims of this application, that is, in the future, a divisional application will be made. Nor does it deny the existence of an invention added by amendment.

  The program according to claim 1, a divided region (for example, a divided region) obtained by dividing a region corresponding to a picture of a video stream according to the number of first decoding means (for example, decoders 565-1 to 565-4 in FIG. 28). A plurality of unit regions (for example, slices) that are processing units of the first decoding unit and the second decoding unit (for example, the decoding unit 601 in FIG. 29) that decode the video stream. A setting step (for example, step S462 in FIG. 31) for setting a divided region including the decoding region, and decoding up to a predetermined middle stage of the unit region in the divided region respectively assigned to the first decoding means. A first half decoding control step (for example, step S464 in FIG. 31) for controlling the first decoding means to be executed in parallel with the first decoding means; In the second half decoding control step of controlling the second decoding means to execute the decoding of the remaining stage of the unit area decoded up to the intermediate stage in parallel with the decoding by the decoding means (for example, FIG. Causing the computer to execute processing including step S509 or S511).

  In the program according to the sixth aspect, the second decoding means can be realized by a GPU (Graphics Processing Unit) (for example, the GPU 521 in FIG. 27).

  The decoding control method according to claim 9, wherein a region corresponding to a picture of a video stream is divided according to the number of first decoding means (for example, decoders 565-1 to 565-4 in FIG. 28) ( For example, a plurality of unit regions (for example, divided decoding regions), which are processing units of the first decoding unit and the second decoding unit (for example, the decoding unit 601 in FIG. 29) that decode the video stream. A setting step (for example, step S462 in FIG. 31) for setting a divided region including a slice), and decoding up to a predetermined intermediate stage of the unit region in the divided region respectively assigned to the first decoding means A first half decoding control step (for example, step S464 in FIG. 31) for controlling the first decoding means to be executed in parallel with the other first decoding means; In the second half decoding control step (for example, FIG. 31), the second decoding means is controlled so as to execute decoding of the remaining stages of the unit area decoded up to the intermediate stage in parallel with decoding by the first decoding means. Step S509 or S511).

  The decoding control device according to claim 10 (for example, the decoding unit 551 in FIG. 28) sets the area corresponding to the picture of the video stream to the number of first decoding means (for example, the decoders 565-1 to 565-4 in FIG. 28). Divided into regions (for example, divided decoding regions), and each processing unit of the first decoding unit and the second decoding unit (for example, the decoding unit 601 in FIG. 29) that decodes the video stream. Setting means (for example, the decoding area setting unit 563 in FIG. 28) for setting a divided area including a plurality of unit areas (for example, slices) and the intermediate stage in parallel with the decoding by the first decoding means Decoding control means (for example, the decoding control unit 564 in FIG. 28) for controlling the second decoding means so as to execute decoding of the remaining stage of the decoded unit area. .

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 2 is a diagram showing an embodiment of an AV (Audio / Visual) processing system 101 to which the present invention is applied. The AV processing system 101 is configured to include an AV (Audio / Visual) processing device 111, a drive 112, external video recording / playback devices 113-1 to 113-n, a mouse 114, and a keyboard 115.

  The AV processing device 111 reproduces, records, displays, and outputs AV (Audio / Visual) data such as a video stream, an audio stream, an AV (Audio / Visual) stream in which the video stream and the audio stream are multiplexed, and still image data. Processes such as decoding and encoding.

  The drive 112 is connected to the interface 129-3 of the AV processing device 111. The drive 112 is connected to a removable medium 116 such as an optical disk, a magneto-optical disk, or a semiconductor memory as appropriate. Various data read from the removable medium 116 are input to the AV processing device 111 or output from the AV processing device 111. Various kinds of data are recorded on the removable medium 116. In addition, the drive 112 is appropriately connected to a removable medium 117 including an optical disk, a magneto-optical disk, or a semiconductor memory, and the program read from the removable medium 117 is installed in the HDD 124.

  The external video recording / reproducing devices 113-1 to 113-n are connected to the interface 129-2 of the AV processing device 111. The external video recording / playback devices 113-1 to 113-n play back AV data such as an AV stream, a video stream, an audio stream, and still image data, and input the played back stream to the AV processing device 111. The external video recording / playback devices 113-1 to 113-n record AV data such as an AV stream, a video stream, an audio stream, and still image data output from the AV processing device 111.

  The external video recording / reproducing devices 113-1 to 113-n are connected to the video special effect audio mixing processing unit 126 of the AV processing device 111. The external video recording / playback apparatuses 113-1 to 113-n input AV data such as an AV stream, a video stream, an audio stream, and still image data to the video special effect audio mixing processing unit 126, and perform various special effects or mixing. The applied AV data is acquired from the video special effect audio mixing processing unit 126.

  Hereinafter, the external video recording / reproducing devices 113-1 to 113-n are simply referred to as the external video recording / reproducing device 113 when it is not necessary to individually distinguish them.

  The mouse 114 and the keyboard 115 are connected to the interface 129-1 of the AV processing device 111.

  The AV processing device 111 includes a CPU (Central Processing Unit) 121, a ROM (Read Only Memory) 122, a RAM (Random Access Memory) 123, a hard disk (HDD) 124, a signal processing unit 125, a video special effect audio mixing processing unit 126, A display 127, a speaker 128, and interfaces (I / F) 129-1 to 129-3 are included. The CPU 121, ROM 122, RAM 123, HDD 124, signal processing unit 125, video special effect audio mixing processing unit 126, and interfaces 129-1 to 129-3 are connected to each other via the bus 130.

  The CPU 121 receives processing instructions and data input by the user using the mouse 114 or the keyboard 115 via the interface 129-1 and the bus 130, and stores them in the ROM 122 based on the input processing instructions. Various processes are executed according to the stored program or the program loaded from the HDD 124 to the RAM 123.

  Further, the CPU 121 performs various types of processing (for example, multiplexing, separation, encoding, decoding, etc.) of AV data such as an AV stream, a video stream, an audio stream, and still image data. For example, the CPU 121 is composed of a multi-core processor and, as will be described later with reference to FIG. 4 and the like, a video encoded by a predetermined method (eg, MPEG2 method) by executing a predetermined program. A stream decoding process is divided into a plurality of threads, and a video stream is decoded in parallel by a plurality of processor cores. Further, as described later with reference to FIG. 11 and the like, the CPU 121 encodes (encodes) a video stream, which is a baseband signal before encoding, by a predetermined method (for example, MPEG2 method).

  The ROM 122 stores basically fixed data among programs used by the CPU 121 and calculation parameters.

  The RAM 123 stores programs used in the execution of the CPU 121 and parameters and data that change as appropriate during the execution.

  For example, the HDD 124 records or reproduces a program executed by the CPU 121 and information.

  The signal processing unit 125 executes various processes (for example, multiplexing, separation, encoding, decoding, etc.) of AV data such as an AV stream, a video stream, an audio stream, and still image data by hardware. Further, the signal processing unit 125 generates a video signal adapted to the standard of the display 127 from the input video stream, and supplies the video signal to the display 127. Further, the signal processing unit 125 generates an audio signal adapted to the standard of the speaker 128 from the input audio stream, and supplies the audio signal to the speaker 128.

  The video special effect audio mixing processing unit 126 converts various special data into AV data such as an AV stream, a video stream, an audio stream, and still image data supplied from the CPU 121, the signal processing unit 125, or the external video recording / playback device 113. Effects (eg, image synthesis, division, conversion, etc.) or mixing (eg, voice synthesis, etc.) are applied. The video special effect audio mixing processing unit 126 supplies AV data subjected to various special effects or mixing to the CPU 121, the signal processing unit 125, or the external video recording / reproducing device 113.

  The display 127 is configured by, for example, a CRT (Cathode Ray Tube), an LCD (Liquid Crystal Display), and the like, and displays an image based on the video signal supplied from the signal processing unit 125.

  The speaker 128 outputs sound based on the sound signal supplied from the signal processing unit 125.

  Hereinafter, an example of processing when the AV processing apparatus 111 encodes or decodes a video stream by the MPEG2 method will be described.

  FIG. 3 is a diagram illustrating an example of a functional configuration of the decoding unit 151 realized by the CPU 121 that executes a predetermined program. The decode unit 151 is configured to include a stream read control unit 161, a stream analysis unit 162, a decode region setting unit 163, a decode control unit 164, decoders 165-1 to 165-4, and a baseband output control unit 166. The

  The stream read control unit 161 controls the drive 112 to read the video stream recorded on the removable medium 116. The stream read control unit 161 acquires the video stream read by the drive 112 via the interface 129-3 and the bus 130. Further, the stream read control unit 161 reads the video stream recorded in the HDD 124 via the bus 130. The stream read control unit 161 temporarily stores the acquired video stream in the stream memory 152 including a cache memory (not shown) in the CPU 121 as necessary. The stream read control unit 161 supplies the acquired video stream to the stream analysis unit 162 or the decoders 165-1 to 165-4 as necessary.

  The stream analysis unit 162 decodes layers up to the picture layer of the video stream, and supplies information obtained by decoding to the decode control unit 164. In addition, the stream analysis unit 162 supplies information indicating the image size (number of pixels in the vertical and horizontal directions of the video stream) obtained by decoding the sequence layer of the video stream to the decoding region setting unit 163.

  As will be described later with reference to FIGS. 4 and 5, the decode area setting unit 163 performs a decoder 165-for each frame of the video stream based on the video size of the video stream and the number of decoders of the decoding unit 151. An area to be divided and decoded by 1 to 165-4 (hereinafter referred to as a divided decoding area) is set. The decode area setting unit 163 supplies information indicating the set divided decode area to the decode control unit 164.

  The decoding control unit 164 is a decoding instruction input by the user using the mouse 114 or the keyboard 115, or each unit of the AV processing device 111 or a decoding input from a functional block different from the decoding unit 151 realized by the CPU 121. The processing of the decoding unit 151 is controlled based on the instruction. The decode control unit 164 supplies information for instructing the supply of the video stream to the stream analysis unit 162 or the decoders 165-1 to 165-4 to the stream read control unit 161. In addition, the decode control unit 164 supplies information instructing the start of decoding to the decoders 165-1 to 165-4.

  Further, the decode control unit 164 acquires information notifying completion of decoding or occurrence of an error from the decoders 165-1 to 165-4. When the decoding control unit 164 obtains information for notifying the occurrence of a decoding error, the decoding control unit 164 determines a position at which decoding is to be resumed, and supplies information for instructing to resume decoding from the decided position to the decoder in which the error has occurred. To do. Further, the decode control unit 164 supplies information indicating the output of the video stream, which is a baseband signal obtained by decoding the video stream, to the baseband output control unit 166.

  The decoders 165-1 to 165-4 decode the divided decoding areas assigned to the decoders in the frames of the video stream in parallel. That is, the decoders 165-1 to 165-4 perform decoding below the slice layer of the video stream. The decoders 165-1 to 165-4 supply the decoded data to the baseband output control unit 166. Hereinafter, the decoders 165-1 to 165-4 are simply referred to as a decoder 165 when it is not necessary to distinguish them individually.

  The baseband output control unit 166 combines the data supplied from the decoders 165-1 to 165-4 to generate a video stream that is a baseband signal after decoding. For example, the baseband output control unit 166 temporarily stores the generated video stream in the baseband memory 153 configured by a cache memory (not shown) in the CPU 121, for example. The baseband output control unit 166 outputs the video stream to the outside (for example, the signal processing unit 125 via the bus 130) based on an instruction from the decode control unit 164.

  Next, the decoding process executed by the decoding unit 151 will be described with reference to the flowcharts of FIGS. This process is started when, for example, the decode control unit 164 acquires a decode instruction input by the user using the mouse 114 or the keyboard 115 via the interface 129-1 and the bus 130. Hereinafter, an example of decoding a video stream recorded on the removable medium 116 will be described.

  In step S1, the decode control unit 164 determines whether or not all video streams have been decoded. If the decoding control unit 164 has not yet decoded all the video streams in the range instructed to be decoded by the user, the decoding control unit 164 determines that all the video streams have not been decoded yet, and the process proceeds to step S2.

  In step S2, the stream analysis unit 162 decodes the sequence layer. Specifically, the decode control unit 164 supplies the stream read control unit 161 with information that instructs reading of the sequence layer of the video stream to be decoded. The drive 112 reads the sequence layer of the video stream to be decoded from the removable medium 116 under the control of the stream read control unit 161, and reads the read data via the interface 129-3 and the bus 130. To supply. The stream read control unit 161 supplies the sequence layer of the video stream to the stream analysis unit 162. The stream analysis unit 162 decodes the sequence layer and supplies information obtained by decoding to the decode control unit 164.

  In step S3, the stream analysis unit 162 detects the image size of the video stream. Specifically, the stream analysis unit 162 detects the image size of the video stream based on the information of the sequence layer of the video stream. The stream analysis unit 162 supplies information indicating the detected image size to the decode region setting unit 163.

  In step S4, the decode area setting unit 163 sets an area to be decoded by each decoder 165. Specifically, first, the decoding area setting unit 163 calculates the number of slices (hereinafter also referred to as the number of divided slices) that each decoder 165 decodes per frame. In the following description, it is assumed that the horizontal width of each slice is equal to the width of the frame (picture), and one slice does not extend between rows.

  The decode area setting unit 163 detects the number of slices included in one frame (picture) of the video stream based on the image size of the video stream. For example, when the vertical image size of the video stream is Spv and the vertical size of the slice is Ssv, the number of slices Nsf per frame is detected by the following equation (1).

  Nsf = Spv ÷ Ssv (1)

  The decode area setting unit 163 calculates the number of divided slices by dividing the detected number of slices per frame by the number of decoders 165. For example, if the number of decoders 165 is Nd, the number of divided slices X is calculated by equation (2).

  X = Nsf ÷ Nd (2)

  Accordingly, the number of slices to be decoded by each decoder 165 is equal, and the amount of data to be decoded per frame is substantially equal.

  For example, when the vertical image size of the video stream is 1088 pixels and the vertical size of the slice is 16 pixels, the number of divided slices X is 17 as follows because the number of decoders 165 is four in this case. It becomes.

Nsf = 1088 ÷ 16 = 68
X = 68 ÷ 4 = 17

  Next, the decode area setting unit 163 sets divided decode areas by dividing the slices in the frame in order from the top for each number of divided slices. For example, as shown in the diagram on the left side of FIG. 6, when 16 slices are included in one frame and decoding is performed in parallel by four decoders, the number of divided slices is 4, so the slices in the frame are A region divided into four in order, that is, a region including slices 181-1 to 181-4, a region including slices 182-1 to 182-4, a region including slices 183-1 to 183-4, and Four areas including the slices 184-1 to 184-4 are set as divided decoding areas.

  Note that, in a video stream encoded by the MPEG2 system, a slice (slice layer) is the lowest layer in a layer where a start position can be detected without adding a start code to the start code. This is the minimum processing unit for decoding a video stream. Therefore, the divided decoding area includes a plurality of slices which are processing units of the decoder 165, and is an area obtained by dividing the area corresponding to the picture of the video stream according to the number of decoders 165. In addition, since consecutive slices on the video stream are arranged in order from the top in the frame (picture), slices in the divided decoding area are continuous on the video stream in order from the top to the bottom of the divided decoding area. To do.

  The decode area setting unit 163 supplies information indicating the set divided decode area to the decode control unit 164.

  In step S5, the decode control unit 164 determines whether all GOPs (Group Of Pictures) have been decoded. The decoding control unit 164 determines whether all GOPs included below the sequence layer decoded in step S2 have been decoded. If it is determined that all GOPs have not been decoded yet, the process proceeds to step Proceed to S6.

  In step S6, the stream analysis unit 162 decodes the GOP layer. Specifically, the decode control unit 164 supplies information that instructs reading of the GOP to be decoded next to the stream read control unit 161. The drive 112 reads the GOP to be decoded next from the removable medium 116 under the control of the stream read control unit 161, and supplies the read GOP to the stream read control unit 161 via the interface 129-3 and the bus 130. To do. The stream read control unit 161 temporarily stores the acquired GOP in the stream memory 152. The stream read control unit 161 supplies the GOP layer of the acquired GOP to the stream analysis unit 162. The stream analysis unit 162 decodes the GOP layer and supplies information obtained by decoding to the decode control unit 164.

  In step S 6, a plurality of GOPs may be read from the removable medium 116 at a time and stored in the stream memory 152.

  In step S7, the decode control unit 164 determines whether all the pictures in the GOP have been decoded. If it is determined that all the pictures in the GOP have not been decoded yet, the process proceeds to step S8.

  In step S8, the stream read control unit 161 reads the picture layer of the next picture to be decoded. Specifically, the decode control unit 164 supplies information that instructs reading of the picture layer of the picture to be decoded next to the stream read control unit 161. The stream read control unit 161 searches the GOP stored in the stream memory 152 for the start position (picture start code) of the next picture to be decoded from the head of the GOP, and decodes it next from the detected start position. The picture layer of the picture to be read is read out. The stream read control unit 161 supplies the read picture layer to the stream analysis unit 162.

  In step S9, the stream analysis unit 162 decodes the picture layer. The stream analysis unit 162 supplies information obtained by decoding the picture layer to the decode control unit 164.

  In step S10, the decode control unit 164 instructs the start of decoding. Specifically, the decode control unit 164 supplies the stream read control unit 161 with information for instructing the supply of each slice included in the divided decode area assigned to each decoder 165 to each decoder 165. In addition, the decode control unit 164 supplies information indicating the start of decoding to each decoder 165.

  In step S11, the stream read control unit 161 detects the start position of the divided decode area. Specifically, the stream read control unit 161 sequentially searches the start position (slice start code) of the slice layer of the picture to be decoded next for each GOP stored in the stream memory 152, and each decoder 165. Is detected, that is, the position at which each decoder 165 starts decoding.

  In step S12, the decoder 165 starts decoding. Specifically, the stream read control unit 161 sequentially reads, from the stream memory 152, slices having the number of divided slices starting from the start position of each divided decode area detected in step S11, that is, slices included in each divided decode area. . The stream read control unit 161 supplies the read slice to the decoder 165 to which the decoding of the divided decoding area is assigned for each divided decoding area. The decoder 165 starts decoding from the first slice in the divided decoding area. The decoder 165 sequentially supplies the decoded data to the baseband output control unit 166. The baseband output control unit 166 stores the acquired data in the baseband memory 153.

  In step S13, the decode control unit 164 determines whether an error has occurred. Specifically, when the information notifying the occurrence of an error is supplied from the decoder 165, the decode control unit 164 determines that an error has occurred, and the process proceeds to step S14.

  In step S14, the decode control unit 164 determines a position at which decoding is resumed. For example, the decode control unit 164 determines a slice next to a slice that has failed to be decoded as a position to resume decoding.

  In step S15, the decoder 165 resumes decoding. Specifically, the decode control unit 164 supplies information instructing restart of decoding from the slice following the slice in which the error has occurred to the decoder 165 in which the error has occurred. The decoder 165 resumes decoding from the designated slice.

  In step S13, the decode control unit 164 determines that no error has occurred when the information notifying the occurrence of the error is not supplied from the decoder 165, the processing of steps S14 and S15 is skipped, and the processing is step. Proceed to S16.

  In step S16, the decode control unit 164 determines whether or not the processing of all the decoders 165 has been completed. Specifically, the decode control unit 164 detects whether or not information notifying the end of decoding of all slices in the divided decode area assigned to each decoder 165 is supplied from all the decoders 165. When the information for notifying the end of decoding of all the slices of the divided decoding area allocated to each decoder 165 is not yet supplied from all the decoders 165, the decode control unit 164 still performs the processing of all the decoders 165. The process returns to step S13, and the process of steps S13 to S16 is repeatedly executed until it is determined in step S16 that the processes of all the decoders 165 have been completed.

  When it is determined in step S16 that the processing of all the decoders 165 has been completed, that is, information for notifying the end of decoding of all the slices in the divided decoding area assigned to each decoder 165 is received from all the decoders 165. If it is supplied to the decode control unit 164, the process proceeds to step S17.

  In step S17, the baseband output control unit 166 outputs data for one frame. Specifically, the decode control unit 164 supplies information for instructing output of data for one frame to the baseband output control unit 166. The baseband output control unit 166 outputs the data for one frame stored in the baseband memory 153 to the outside (for example, the signal processing unit 125 via the bus 130). Thereafter, the process returns to step S7.

  If it is determined in step S7 that all the pictures in the GOP have not been decoded yet, the process proceeds to step S8, and the processes after step S8 described above are executed. That is, the next picture is decoded.

  If it is determined in step S7 that all the pictures in the GOP have been decoded, the process returns to step S5.

  If it is determined in step S5 that not all GOPs have been decoded yet, the process proceeds to step S6, and the processes after step S6 described above are executed. That is, the next GOP is decoded.

  If it is determined in step S5 that all GOPs have been decoded, the process returns to step S1.

  If it is determined in step S1 that the entire video stream has not been decoded yet, the process proceeds to step S2, and the processes after step S2 described above are executed. That is, the next sequence layer of the video stream is decoded.

  If it is determined in step S1 that the entire video stream has been decoded, the decoding process ends.

  As described above, since it is only necessary to detect the start position of each divided decode area as the position where each decoder 165 starts decoding for each frame (picture), the time for detecting the position where each decoder 165 starts decoding is determined. This shortens the video stream decoding speed.

  Next, another embodiment for speeding up decoding of a video stream will be described with reference to FIGS.

  FIG. 7 is a diagram illustrating an example of a functional configuration of the encoding unit 201 realized by the CPU 121 that executes a predetermined program. The encoding unit 201 includes an encoder 211, a picture count unit 212, a slice count unit 213, a start position memory 214, a start position recording control unit 215, and a stream output control unit 216.

  The encoder 211 is an instruction for encoding input by the user using the mouse 114 or the keyboard 115, or an instruction for encoding input from each unit of the AV processing device 111 or from a functional block different from the encoding unit 201 implemented in the CPU 121. The video stream, which is a baseband signal before encoding, input from the outside is encoded (encoded) based on the MPEG2 system, and the encoded data is supplied to the stream output control unit 216.

  The encoder 211 supplies information notifying the start of picture encoding to the picture count unit 212 and the start position recording control unit 215 every time encoding of a picture (picture layer) is started. Further, the encoder 211 causes the start position memory 214 to record information indicating the start position of each frame (picture) of the encoded video stream (hereinafter referred to as picture start position information). The encoder 211 supplies information for notifying the end of picture encoding to the slice count unit 213 and the start position recording control unit 215 every time encoding of each picture is completed. Further, the encoder 211 records information indicating the data length of the encoded picture (hereinafter referred to as picture data length information) in the start position memory 214.

  Further, every time encoding of a slice (slice layer) is started, the encoder 211 supplies information notifying start of encoding of the slice to the slice count unit 213 and the start position recording control unit 215. Further, the encoder 211 records information indicating the start position of the encoded video stream (hereinafter referred to as slice start position information) in the start position memory 214. The encoder 211 supplies information for notifying the end of the encoding of the slice to the start position recording control unit 215 every time the encoding of each slice is completed. Furthermore, the encoder 211 causes the start position memory 214 to record information indicating the data length of the encoded slice (hereinafter referred to as slice data length information).

  The picture count unit 212 counts the number of pictures in a clip, which is a unit for managing a video stream on a recording medium. Specifically, the picture count unit 212 manages the picture counter, and increments the value of the picture counter every time information that notifies the start of picture encoding is supplied from the encoder 211. The picture count unit 212 supplies information indicating the value of the picture counter to the start position recording control unit 215.

  The slice count unit 213 counts the number of slices in the picture. Specifically, the slice count unit 213 manages the slice counter, and increments the value of the slice counter every time information that notifies the start of encoding of the slice is supplied from the encoder 211. Also, the slice count unit 213 resets the value of the slice counter when information that notifies the end of picture encoding is supplied from the encoder 211.

  The start position recording control unit 215 reads picture start position information, slice start position information, picture data length information, or slice data length information recorded in the start position memory 214. In addition, the start position recording control unit 215 acquires information indicating the value of the picture counter from the picture count unit 212. Further, the start position recording control unit 215 acquires information indicating the value of the slice counter from the slice count unit 213.

  The start position recording control unit 215 records information indicating the position of each picture (hereinafter referred to as picture position information) in the video stream based on the value of the picture counter, the picture start position information, and the picture data length information. By supplying the instructed information to the stream output control unit 216, the recording of the picture position information is controlled. The start position recording control unit 215 also applies information indicating the position of each slice (hereinafter referred to as slice position information) to the video stream based on the value of the slice counter, slice start position information, and slice data length information. By supplying information for instructing recording to the stream output control unit 216, recording of slice position information is controlled.

  The stream output control unit 216 temporarily stores the video stream encoded by the encoder 211 in the stream memory 202 configured by, for example, a cache memory in the CPU 121. Based on the instruction from the start position recording control unit 215, the stream output control unit 216 records picture position information in a user data (User Data) area of the GOP layer corresponding to each GOP of the encoded video stream, for example. Note that the picture position information corresponding to all GOPs included in the sequence layer and below is recorded in the user data (User Data) area of the extension and user area of the sequence layer of the video stream. Also good. Also, the stream output control unit 216, based on the instruction of the start position recording control unit 215, for example, a user in the extension and user (Extension and User) area of the picture layer corresponding to each picture of the encoded video stream Record slice position information in the data (User Data) area.

  The stream output control unit 216 reads the video stream stored in the stream memory 202, and outputs the read video stream to the outside (for example, the drive 112 or the HDD 124 via the bus 130).

  FIG. 8 is a diagram illustrating an example of a functional configuration of the decoding unit 251 realized by the CPU 121 that executes a predetermined program. The decoding unit 251 is different from the decoding unit 151 in FIG. 3 in that a start position detection unit 271 is provided instead of the decoding region setting unit 163. In the figure, the parts corresponding to those in FIG. 3 are given the same reference numerals in the last two digits, and the description of the parts having the same processing will be omitted because it will be repeated.

  The decoding unit 251 includes a stream read control unit 261, a stream analysis unit 262, a decode control unit 264, decoders 265-1 to 265-4, a baseband output control unit 266, and a start position detection unit 271. The

  In addition to the processing of the stream analysis unit 162 in FIG. 3, the stream analysis unit 262 extracts picture position information and slice position information recorded in the video stream, and starts position detection of the extracted picture position information and slice position information. To the unit 271.

  In addition to the processing of the decode control unit 164 in FIG. 3, the decode control unit 264 instructs information to detect the start position of the picture designated by the picture number (FIG. 9) (hereinafter referred to as start position detection instruction information). Alternatively, start position detection instruction information for instructing detection of the start position of the slice designated by the slice number (FIG. 10) is supplied to the start position detection unit 271.

  The start position detection unit 271 detects the start position of the picture or slice specified by the start position detection instruction information based on the picture position information or the slice position information. The start position detection unit 271 supplies information indicating the start position of the detected picture or slice to the decode control unit 264.

  Hereinafter, the decoders 265-1 to 265-4 are simply referred to as a decoder 265 when it is not necessary to distinguish them individually.

  FIG. 9 is a diagram illustrating an example of a data structure of picture position information. The picture position information is recorded for each GOP in the video stream, and is configured to include a picture start address and individual information corresponding to each picture included in the GOP.

  The picture start address indicates a relative position (relative address) from the start of the clip at the start position of the first picture in the GOP. Note that when a video stream is recorded on a recording medium (for example, the removable medium 116 or the HDD 124), the picture start address may be the first address of the position where the first picture in the GOP is recorded on the recording medium. Good.

  The individual information is configured to include a picture number, a data length, and an offset.

  The picture number is a serial number counted from the first picture in the clip. Since the picture number is not reset in one clip, it indicates the number of the picture from the beginning of the clip.

  The data length indicates the data length of the picture.

  The offset indicates an offset (relative address) from the picture start address. Accordingly, based on the picture start address and the offset, a relative address from the beginning of the clip at the start position of each picture can be obtained. In addition, when recording a video stream on a recording medium (for example, the removable medium 116 or the HDD 124), when recording a video stream on a recording medium, for each picture, instead of an offset, the position of the picture is recorded on the recording medium. The head address may be recorded.

  FIG. 10 is a diagram illustrating an example of a data configuration of slice position information. The slice position information is recorded for each picture in the video stream, and is configured to include a slice start address and individual information corresponding to each slice included in the picture.

  The slice start address indicates a relative address from the start of the clip at the start position of the first slice in the picture. Note that the slice start address may be the head address of the position where the head slice in the picture is recorded on the recording medium (for example, the removable medium 116 or the HDD 124).

  The individual information is configured to include a slice number, a data length, and an offset.

  The slice number is a serial number counted starting from the first slice in the picture. Since the slice number is reset every time the picture changes, it indicates the number of the slice from the beginning in the picture.

  The data length indicates the data length of the slice.

  The offset indicates an offset (relative address) from the slice start address. Therefore, based on the slice start address and the offset, the relative address from the beginning of the clip at the start position of each slice can be obtained. When recording a video stream on a recording medium (for example, the removable medium 116 or the HDD 124), for each slice, an address of a position at which the slice is recorded on the recording medium (for example, the removable medium 116 or the HDD 124) instead of the offset. May be recorded.

  Next, the encoding process executed by the encoding unit 201 will be described with reference to the flowchart of FIG. This process is started when, for example, the encoder 211 acquires an encoding instruction input by the user using the mouse 114 or the keyboard 115 via the interface 129-1 and the bus 130. Hereinafter, a video stream that is a baseband signal before encoding is input from the external video recording apparatus 113-1 to the encoder 211 via the interface 129-2 and the bus 130, and the encoded video stream is input to the removable medium 116. An example of recording will be described.

  In step S101, the encoder 211 determines whether all video streams have been encoded. If it is determined that all the video streams are not yet encoded, for example, if the input of the video streams from the external video recording device 113-1 is continued, the process proceeds to step S102.

  In step S102, the encoder 211 determines whether to change the sequence layer. If it is determined that the sequence layer is to be changed, for example, if the image size of the video stream is changed or information for instructing the change of the sequence layer is input from the outside to the encoder 211, the process proceeds to step S103.

  In step S103, the encoder 211 encodes the sequence layer of the video stream. The encoder 211 supplies the encoded data to the stream output control unit 216. The stream output control unit 216 stores the acquired data in the stream memory 202.

  If it is determined in step S102 that the sequence layer is not changed, the process of step S103 is skipped, and the process proceeds to step S104.

  In step S104, the encoder 211 encodes the GOP layer. At this time, the encoder 211 reserves an area for later recording picture position information in the user data area of the GOP layer. The encoder 211 supplies the encoded data to the stream output control unit 216. The stream output control unit 216 stores the acquired data in the stream memory 202.

  In step S105, the encoding unit 201 performs an encoding process below the picture layer. The details of the encoding process below the picture layer will be described later with reference to FIG. Thereafter, the processing returns to step S101, and the processing of steps S101 to S105 is repeatedly executed until it is determined in step S101 that the video stream has been encoded, and the video stream is encoded.

  If it is determined in step S101 that the video stream has been encoded, for example, if the input of the video stream from the external video recording device 113-1 has been completed and the encoding of the input video stream has been completed, The encoding process ends.

  Next, the details of the encoding processing below the picture layer in step S105 of FIG.

  In step S121, the encoder 211 determines whether all pictures in the GOP have been encoded. The encoder 211 checks whether or not the number of pictures to be included in the currently encoded GOP have been encoded. If the number of pictures to be included in the currently encoded GOP has not yet been encoded, It is determined that not all pictures have been encoded, and the process proceeds to step S122.

  In step S122, the picture count unit 212 increments the picture counter. Specifically, the encoder 211 supplies information notifying the start of encoding of the next picture to the picture count unit 212. The picture count unit 212 increments the picture counter.

  In step S123, the encoder 211 determines the start position of the picture. Specifically, the encoder 211 determines a position where recording of a picture to be encoded next is started. When the next picture to be encoded is the first picture of the GOP, the encoder 211 stores, in the start position memory 214, picture start position information indicating a relative address from the start of the clip at the determined start position. If the next picture to be encoded is not the first picture of the GOP, the encoder 211 stores picture start position information indicating an offset from the first picture of the GOP at the determined start position in the start position memory 214.

  In step S124, the stream output control unit 216 records the start position of the picture. Specifically, the encoder 211 supplies information that notifies the start of encoding of the next picture to the start position recording control unit 215. The start position recording control unit 215 acquires information indicating the value of the picture counter from the picture count unit 212 and acquires picture start position information from the start position memory 214.

  When the next picture to be encoded is the first picture of the GOP, the stream output control unit 216 corresponds to the GOP being encoded stored in the stream memory 202 under the control of the start position recording control unit 215. For picture position information, the value of picture start position information is recorded at the picture start address. Also, the stream output control unit 216 records the value of the picture counter in the picture No. for the individual information at the head of the picture position information corresponding to the GOP being encoded under the control of the start position recording control unit 215, Record 0 for the offset.

  When the next picture to be encoded is not the first picture of the GOP, the stream output control unit 216 encodes next to the picture position information corresponding to the GOP being encoded under the control of the start position recording control unit 215. For individual information corresponding to a picture, the value of the picture counter is recorded in the picture No., and the value of the picture start position information is recorded in the offset.

  In step S125, the encoder 211 encodes the picture layer. At this time, the encoder 211 reserves an area for later recording slice position information in the user data area of the picture layer. The encoder 211 supplies the encoded data to the stream output control unit 216. The stream output control unit 216 stores the acquired data in the stream memory 202.

  In step S126, the encoder 211 determines whether all slices in the picture have been encoded. The encoder 211 checks whether or not a predetermined number of slices included in the picture have been encoded. If the predetermined number of slices included in the picture has not yet been encoded, all the slices in the picture have not been encoded yet. It is determined that encoding has not been performed, and the process proceeds to step S127.

  In step S127, the slice count unit 213 increments the slice counter. Specifically, the encoder 211 supplies information notifying the start of encoding of the next slice to the slice count unit 213. The slice count unit 213 increments the slice counter.

  In step S128, the encoder 211 determines the start position of the slice. Specifically, the encoder 211 determines a position where recording of a slice to be encoded next is started. When the slice to be encoded next is the first slice of the picture, the encoder 211 stores slice start position information indicating the relative address from the start of the clip at the determined start position in the start position memory 214. Also, if the slice to be encoded next is not the first slice of the picture, the encoder 211 stores slice start position information indicating an offset from the first slice of the picture at the determined start position in the start position memory 214.

  In step S129, the stream output control unit 216 records the start position of the slice. Specifically, the encoder 211 supplies information that notifies the start of encoding of the next slice to the start position recording control unit 215. The start position recording control unit 215 acquires information indicating the value of the slice counter from the slice count unit 213, and acquires slice start position information from the start position memory 214.

  When the next slice to be encoded is the first slice of the picture, the stream output control unit 216 corresponds to the picture being encoded stored in the stream memory 202 under the control of the start position recording control unit 215. For slice position information, the value of slice start position information is recorded at the slice start address. Also, the stream output control unit 216 records the value of the slice counter in the slice number for the individual information at the beginning of the slice position information corresponding to the picture being encoded, based on the control of the start position recording control unit 215. Record 0 in the offset.

  If the next slice to be encoded is not the first slice of the picture, the stream output control unit 216 performs the next encoding in the slice position information corresponding to the picture being encoded, under the control of the start position recording control unit 215. For the individual information corresponding to the slice to be recorded, the slice counter value is recorded in the slice number, and the slice start position information value is recorded in the offset.

  In step S130, the encoder 211 encodes layers below the slice layer. The encoder 211 supplies the encoded data to the stream output control unit 216. The stream output control unit 216 stores the acquired data in the stream memory 202.

  In step S131, the stream output control unit 216 records the data length of the slice. Specifically, the encoder 211 causes the start position memory 214 to store slice data length information indicating the data length of the slice that has been encoded. The encoder 211 supplies information indicating that the encoding of the slice is completed to the start position recording control unit 215. The start position recording control unit 215 acquires slice data length information from the start position memory 214. Based on the control of the start position recording control unit 215, the stream output control unit 216 performs individual information corresponding to a slice that has been encoded within the slice position information corresponding to the picture being encoded stored in the stream memory 202. For, record the value of slice data length information in the data length.

  Thereafter, the processing returns to step S126, and the processing of steps S126 to S131 is repeatedly executed until it is determined in step S126 that all slices in the picture are encoded, and all slices in the picture are encoded.

  In step S126, when the encoding of a predetermined number of slices included in the picture has been completed, the encoder 211 determines that all the slices in the picture have been encoded, and the process proceeds to step S132.

  In step S132, the stream output control unit 216 records the data length of the picture. Specifically, the encoder 211 causes the start position memory 214 to store picture data length information indicating the data length of a picture that has been encoded. Also, the encoder 211 supplies information indicating that the encoding of the picture has been completed to the start position recording control unit 215. The start position recording control unit 215 acquires picture data length information from the start position memory 214. Based on the control of the start position recording control unit 215, the stream output control unit 216 performs individual information corresponding to a picture that has been encoded within the picture position information corresponding to the GOP being encoded stored in the stream memory 202. Is recorded with the value of the picture data length information in the data length.

  In step S133, the slice count unit 213 resets the slice counter. Specifically, the encoder 211 supplies the slice count unit 213 with information indicating that picture encoding has been completed. The slice count unit 213 resets the slice counter.

  Thereafter, the processing returns to step S121, and in step S121, the processing in steps S121 to S133 is repeatedly executed until it is determined that all the pictures in the GOP are encoded, and all the pictures in the GOP are encoded.

  In step S121, when encoding of the number of pictures to be included in the currently encoded GOP has been completed, the encoder 211 determines that all the pictures in the GOP have been encoded, and the process proceeds to step S134.

  In step S134, the stream output control unit 216 outputs the video stream, and the encoding processing below the picture layer ends. Specifically, for example, when the data amount of the stream memory 202 exceeds a predetermined threshold, the stream output control unit 216 converts the encoded video stream stored in the stream memory 202 into the bus 130 and It is supplied to the drive 112 via the interface 129-3. The drive 112 records the video stream on the removable medium 116.

  Next, a decoding process in the case where the video stream encoded by the encoding unit 201 is decoded by the decoding unit 251 will be described with reference to the flowcharts of FIGS. This process is started when, for example, the decode control unit 264 acquires a decode instruction input by the user using the mouse 114 or the keyboard 115 via the interface 129-1 and the bus 130. Hereinafter, an example of decoding a video stream recorded on the removable medium 116 will be described. Further, hereinafter, as shown on the right side of FIG. 1 described above, for each picture, the decoders 265 sequentially decode the slices in order from the top slice in the picture.

  In step S151, it is determined whether or not the entire video stream has been decoded by the same processing as in step S1 of FIG. 4 described above. If it is determined that the video stream has not been completely decoded, the process proceeds to step S152.

  In step S152, the sequence layer of the video stream to be decoded is read from the removable medium 116 by the same process as in step S2 of FIG. 4 described above, and the sequence layer is decoded.

  In step S153, it is determined whether or not all GOPs have been decoded by the same processing as in step S5 of FIG. 4 described above. If it is determined that not all GOPs have been decoded yet, the process proceeds to step S154.

  In step S154, the GOP to be decoded next is read from the removable medium 116 by the same processing as in step S6 of FIG. 4 described above, and the GOP layer of the read GOP is decoded.

  In step S155, the stream analysis unit 262 extracts picture position information. Specifically, the stream analysis unit 262 extracts the picture position information recorded in the user data area of the GOP layer decoded in step S154. The stream analysis unit 262 supplies the extracted picture position information to the start position detection unit 271.

  In step S156, it is determined whether or not all the pictures in the GOP have been decoded by the same processing as in step S7 of FIG. 4 described above. If it is determined that all the pictures in the GOP have not been decoded yet, the process proceeds to step S157.

  In step S157, the stream read control unit 261 reads the picture layer of the picture to be decoded next. Specifically, the decode control unit 264 supplies start position detection instruction information for instructing detection of the start position of the next picture to be decoded to the start position detection unit 271 by designating the picture number. The start position detector 271 detects the start position of the picture of the designated picture number based on the picture position information. The start position detection unit 271 supplies information indicating the detected start position of the picture to the decode control unit 264. The decode control unit 264 supplies the stream read control unit 261 with information instructing to read the picture layer from the start position of the picture detected by the start position detection unit 271. The stream read control unit 261 reads the picture layer of the next picture to be decoded starting from the designated start position from the GOP stored in the stream memory 152. The stream read control unit 261 supplies the read picture layer to the stream analysis unit 262.

  In step S158, the picture layer is decoded by the same processing as in step S9 of FIG. 4 described above.

  In step S159, the stream analysis unit 262 extracts slice position information. Specifically, the stream analysis unit 262 extracts slice position information recorded in the user data area of the picture layer decoded in step S158. The stream analysis unit 262 supplies the extracted slice position information to the start position detection unit 271.

  In step S160, the start position detector 271 detects a position where decoding is to start. Specifically, the decode control unit 264 supplies start position detection instruction information for instructing detection of the start position of each slice to be decoded next by each decoder 265 to the start position detection unit 271 by designating a slice number. To do. The start position detector 271 detects the start position of the slice of the designated slice number based on the slice position information. The decode control unit 264 supplies information indicating the detected start position of each slice to the decode control unit 264.

  In step S161, the decode control unit 264 instructs to start decoding. Specifically, the decode control unit 264 supplies the stream read control unit 261 with information instructing supply to each decoder 265 of each slice starting from the start position detected in step S160. In addition, the decoding control unit 264 supplies information indicating the start of decoding of the next slice to each decoder 265.

  In step S162, the decoder 265 starts decoding. Specifically, the stream read control unit 261 reads a slice starting from the start position instructed by the decode control unit 264 from the stream memory 152 and supplies the read slice to each decoder 265. The decoder 265 starts decoding the acquired slice. When the decoding of the slice started in step S162 is completed, the decoder 265 supplies information indicating that the decoding is completed to the decode control unit 264, and supplies the decoded data to the baseband output control unit 266. The baseband output control unit 266 stores the acquired data in the baseband memory 153.

  In step S163, whether or not an error has occurred is determined by the same processing as in step S13 of FIG. If it is determined that no error has occurred, the process proceeds to step S164.

  In step S164, the decode control unit 264 determines whether there is a decoder 265 that has been decoded. If the decoding control unit 264 has not obtained any information indicating that decoding has been completed from any decoder 165, the decoding control unit 264 determines that there is no decoder 265 that has completed decoding, and the process returns to step S163. In step S163, an error is detected. Until it is determined that the error has occurred or it is determined in step S164 that there is a decoder 265 that has finished decoding, the determination processing in steps S163 and S164 is repeatedly executed.

  In step S164, if the decoding control unit 264 has acquired information indicating that decoding has been completed from one or more decoders 165, the decoding control unit 264 determines that there is a decoder 265 that has completed decoding, and the process proceeds to step S165. .

  In step S165, the decoding control unit 264 determines whether or not the decoder 265 that has finished decoding has decoded all the slices allocated in the picture being decoded. If it is determined that all the allocated slices in the picture being decoded have not been decoded yet, the process returns to step S160, and the processes after step S160 are executed. That is, the decoder 265 that has finished decoding decodes the next slice assigned in the picture being decoded.

  If it is determined in step S163 that an error has occurred, the process proceeds to step S166.

  In step S166, the decoding control unit 264 determines a position at which decoding is resumed. For example, the decoding control unit 264 determines the next slice of the slice that has failed to be decoded as the decoding restart position. Thereafter, the process returns to step S160, and the processes after step S160 are executed. That is, the decoder 265 in which the error has occurred decodes the slice next to the slice in which the error has occurred.

  If it is determined in step S165 that the decoder 265 that has finished decoding has decoded all the slices allocated in the picture being decoded, the process proceeds to step S167.

  In step S167, the decode control unit 264 determines whether or not the processing of all the decoders 265 has been completed. Specifically, the decoding control unit 264 detects whether there is a decoder 265 that has not finished decoding the assigned slice in the picture being decoded, and has not finished decoding the assigned slice. If there is a decoder 265, it is determined that the processing of all the decoders 265 has not been completed yet, the processing returns to step S163, and the processing after step S163 is executed. That is, the decoder 265 that has finished decoding the allocated slice in the picture being decoded is in a standby state, and the processing of the decoder 265 that has not finished decoding the allocated slice is continuously executed.

  If it is determined in step S167 that all the decoders 265 have been processed, the process proceeds to step S168.

  In step S168, data for one frame is output by the same process as in step S17 of FIG. 5 described above, and the process returns to step S156.

  If it is determined in step S156 that all the pictures in the GOP have not been decoded yet, the process proceeds to step S157, and the processes after step S157 described above are executed. That is, the next picture is decoded.

  If it is determined in step S156 that all the pictures in the GOP have been decoded, the process returns to step S153.

  If it is determined in step S153 that not all GOPs have been decoded yet, the process proceeds to step S154, and the processes after step S154 described above are executed. That is, the next GOP is decoded.

  If it is determined in step S153 that all GOPs have been decoded, the process returns to step S151.

  If it is determined in step S151 that the entire video stream has not been decoded yet, the process proceeds to step S152, and the processes after step S152 described above are executed. That is, the next sequence layer of the video stream is decoded.

  If it is determined in step S151 that the entire video stream has been decoded, the decoding process ends.

  Next, referring to the flowchart of FIG. 15, the decoding start position detection process executed by the decoding unit 251 when decoding is started from a picture specified by a user instruction or the like without decoding the video stream from the beginning. Will be explained.

  In step S181, the decode control unit 164 detects a position where decoding is started. Specifically, the decoding control unit 164 detects the picture number of the picture at the decoding start position designated by the user.

  In step S182, the sequence layer is decoded by the same processing as in step S2 of FIG. 4 described above.

  In step S183, the GOP layer of the GOP including the picture to be decoded is decoded by the same processing as in step S6 of FIG. 4 described above.

  In step S184, GOP picture position information including a picture to be decoded is extracted by the same processing as in step S155 of FIG. 13 described above.

  In step S185, the picture layer of the picture to be decoded is read based on the picture position information by the same process as in step S157 of FIG.

  In step S186, the picture layer of the picture to be decoded is decoded by the same processing as in step S158 of FIG. 13 described above.

  In step S187, slice information of a picture to start decoding is extracted by the same processing as in step S159 of FIG.

  In step S188, the position of the slice at which each decoder 265 starts decoding is detected based on the slice position information by the same processing as in step S160 of FIG. 13 described above.

  In step S189, information instructing supply to each decoder 265 of each slice starting from the start position detected in step S188 is supplied to the stream read control unit 261 by the same processing as in step S161 of FIG. Each decoder 265 is instructed to start decoding a slice starting from the start position detected in step S188, and the decoding start position detection process ends.

  In this way, when the stream analysis unit 262 decodes the picture layer, the decoding start position can be quickly detected based on the picture position information without searching for the start position (start code) of the picture layer on the video stream. Is done. Further, when each decoder 465 decodes a slice, the start position of decoding is quickly detected based on the slice position information without searching for the start position (start code) of the slice layer on the video stream. Therefore, the time for detecting the position to start decoding is shortened, and the decoding of the video stream can be speeded up. Similarly, when decoding is performed by one decoder without dividing the decoding process, the time for detecting the position to start decoding is shortened, and the decoding of the video stream can be speeded up.

  Note that the picture position information and the slice position information can be recorded in a file different from the video stream without being recorded in the video stream. With reference to FIGS. 16 to 23, an embodiment in which picture position information and slice position information are recorded in a file different from a video stream will be described.

  FIG. 16 is a diagram illustrating an example of a functional configuration of the encoding unit 301 realized by the CPU 121 that executes a predetermined program. The encoding unit 301 is different from the encoding unit 201 in FIG. 7 in that a GOP count unit 321 is provided. In the figure, the parts corresponding to those in FIG. 7 are given the same reference numerals in the last two digits, and the description of the parts having the same processing will be omitted because it will be repeated.

  The encoding unit 301 includes an encoder 311, a picture count unit 312, a slice count unit 313, a start position memory 314, a start position recording control unit 315, a stream output control unit 316, and a GOP count unit 321.

  In addition to the processing of the encoder 211 in FIG. 7, the encoder 311 sends the clip of the video stream generated by encoding to the recording medium (for example, the removable medium 116 or the HDD 124) before starting the encoding of the video stream. Information indicating the recording start position (hereinafter referred to as clip start position information) is stored in the start position memory 314. In addition, when the encoding of the video stream is started, the encoder 311 supplies information for notifying the start of the encoding of the video stream to the start position recording control unit 315. Further, every time encoding of the GOP (GOP layer) is started, the encoder 311 supplies information notifying the start of encoding of the GOP to the GOP count unit 321 and the start position recording control unit 315.

  The GOP count unit 321 counts the number of GOPs in the clip. Specifically, the GOP count unit 321 manages the GOP counter, and increments the value of the GOP counter every time information that notifies the start of GOP encoding is supplied from the encoder 311.

  The start position recording control unit 315 reads picture start position information, slice start position information, picture data length information, slice data length information, or clip start position information recorded in the start position memory 314. In addition, the start position recording control unit 315 acquires information indicating the value of the picture counter from the picture count unit 312. Further, the start position recording control unit 315 acquires information indicating the value of the slice counter from the slice count unit 313. Also, the start position recording control unit 315 acquires information indicating the value of the GOP counter from the GOP count unit 321.

  The start position recording control unit 315 generates information indicating the position of each picture and each slice in the clip (hereinafter referred to as clip position information). The start position recording control unit 315 temporarily stores the clip position information in the stream memory 202, and updates the clip position information as appropriate.

  The stream output control unit 316 temporarily stores the video stream encoded by the encoder 311 in the stream memory 202. The stream output control unit 316 reads the video stream or clip position information stored in the trim memory 202, and externally reads the read video stream or clip position information (for example, the drive 112 or the HDD 124 via the bus 130). Output.

  FIG. 17 is a diagram illustrating an example of a functional configuration of the decoding unit 351 realized by the CPU 121 that executes a predetermined program. In the figure, portions corresponding to those in FIG. 8 are denoted by the same reference numerals in the last two digits, and the description of the portions having the same processing will be omitted because it will be repeated.

  The decoding unit 351 includes a stream read control unit 361, a stream analysis unit 362, a decode control unit 364, decoders 365-1 to 365-4, a baseband output control unit 366, and a start position detection unit 371. Composed.

  In addition to the processing of the decode control unit 264 of FIG. 8, the decode control unit 364 supplies information for instructing acquisition of clip position information corresponding to the clip of the video stream to be decoded to the start position detection unit 371.

  The start position detection unit 371 reads clip position information corresponding to the clip of the video stream to be decoded from the recording medium on which the clip is recorded. The start position detection unit 371 detects the start position of the picture or slice specified by the start position detection instruction information from the decode control unit 364 based on the clip position information. The start position detection unit 371 supplies information indicating the start position of the detected picture or slice to the decode control unit 364.

  Hereinafter, when it is not necessary to distinguish the decoders 365-1 to 365-4 from each other, they are simply referred to as a decoder 365.

  FIG. 18 is a diagram illustrating an example of a data structure of clip position information. The clip position information is recorded for each clip of the video stream, and is configured to include a clip number, a clip start address, picture position information, and slice position information.

  The clip number is a number for identifying a clip corresponding to the clip position information, and is a number assigned for uniquely identifying the clip of the video stream recorded on the recording medium.

  The clip start address indicates the start address of the position where the clip is recorded on the recording medium (for example, the removable medium 116 or the HDD 124).

  As the picture position information, as will be described later with reference to FIG. 19, information indicating the position of each picture in the GOP is recorded for each GOP included in the clip.

  As described later with reference to FIG. 20, the slice position information records information indicating the position of each slice in the picture for each picture included in the clip.

  FIG. 19 is a diagram illustrating an example of a data structure of picture position information included in the clip position information of FIG. The picture position information of the clip position information is different from the picture position information of FIG. 10 in that a GOP No. is added. The GOP No. is a number assigned to uniquely identify the GOP in the clip, and indicates the GOP No. of the GOP corresponding to the picture position information.

  FIG. 20 is a diagram illustrating an example of a data configuration of slice position information included in the clip position information of FIG. The slice position information of the clip position information is different from the slice position information of FIG. 11 in that a picture number is added. The picture number indicates the picture number of the picture corresponding to the slice position information.

  Next, the encoding process executed by the encoding unit 301 will be described with reference to the flowchart of FIG. This process is started when, for example, the encoder 311 acquires an encoding instruction input by the user using the mouse 114 or the keyboard 115 via the interface 129-1 and the bus 130. Hereinafter, a video stream that is a baseband signal before encoding is input from the external video recording apparatus 113-1 to the encoder 311 via the interface 129-2 and the bus 130, and the encoded video stream is input to the removable medium 116. An example of recording will be described.

  In step S201, the encoder 311 determines the start position of the clip. Specifically, the encoder 311 determines a position where recording of the clip of the video stream to be encoded onto the removable medium 116 is started. The encoder 311 stores clip start position information indicating the determined start position in the start position memory 314.

  In step S202, the start position recording control unit 315 records the start position of the clip. Specifically, the encoder 311 supplies information that notifies the start of encoding of the video stream to the start position recording control unit 315. The start position recording control unit 315 acquires clip start position information from the start position memory 314. The start position recording control unit 315 generates clip position information. The start position recording control unit 315 records the clip number of the clip of the video stream to start encoding in the clip position information. In addition, the start position recording control unit 315 records the value of the clip start position information at the clip start address of the clip position information. The start position recording control unit 315 stores the clip position information in the stream memory 152.

  In step S203, it is determined whether or not the entire video stream has been encoded by the same processing as in step S101 of FIG. 11 described above. If it is determined that not all video streams have been encoded yet, the process proceeds to step S204.

  In step S204, whether or not to change the sequence layer is determined by the same processing as in step S102 of FIG. If it is determined to change the sequence layer, the process proceeds to step S205.

  In step S205, the sequence layer is encoded by the same process as in step S103 of FIG.

  If it is determined in step S204 that the sequence layer is not changed, the process of step S205 is skipped, and the process proceeds to step S206.

  In step S206, the GOP count unit 321 increments the GOP counter. Specifically, the encoder 311 supplies information notifying the start of encoding of the next GOP to the GOP count unit 321. The GOP count unit 321 increments the GOP counter.

  In step S207, the start position recording control unit 315 records the GOP No. Specifically, the encoder 311 supplies information for notifying the start of encoding of the next GOP to the start position recording control unit 315. The start position recording control unit 315 acquires information indicating the value of the GOP counter from the GOP count unit 321. The start position recording control unit 315 records the value of the GOP counter in the GOP No. for the picture position information corresponding to the GOP to be encoded next to the clip position information stored in the stream memory 202.

  In step S208, the GOP layer is encoded by the same processing as in step S104 of FIG. 11 described above.

  In step S209, the encoding processing below the picture layer described above with reference to FIG. 12 is performed. Note that, unlike the encoding processing below the picture layer executed by the encoder 211 in FIG. 7, picture position information and slice position information are recorded in clip position information stored in the stream memory 202.

  Thereafter, the process returns to step S203, and the processes of steps S203 to S209 are repeatedly executed until it is determined in step S203 that all the video streams have been encoded.

  If it is determined in step S203 that the entire video stream has been encoded, the process proceeds to step S210.

  In step S210, the stream output control unit 316 outputs clip position information, and the encoding process ends. Specifically, the stream output control unit 316 reads the clip position information stored in the stream memory 202, and supplies the read clip position information to the drive 112 via the bus 130 and the interface 129-3. . The drive 112 records the clip position information on the removable medium 116 as a file different from the corresponding clip.

  Next, the decoding process executed by the decoding unit 351 will be described with reference to the flowcharts of FIGS. In the flowcharts of FIGS. 22 and 23, the process of step S251 is added as compared with the flowcharts of FIGS. 13 and 14 described above.

  That is, in step S251, the start position detection unit 371 acquires clip position information. Specifically, the decode control unit 364 supplies information for instructing acquisition of clip position information corresponding to the clip of the video stream to be decoded to the start position detection unit 371. The drive 112 reads the clip position information from the removable medium 116 under the control of the start position detection unit 371, and supplies the read clip position information to the start position detection unit 371 via the interface 129-3 and the bus 130. To do.

  Further, the flowcharts of FIGS. 22 and 23 do not need to extract the picture position information and the slice position information recorded in the video stream as compared with the flowcharts of FIGS. 13 and 14 described above. Processing corresponding to steps S155 and S159 is deleted. Therefore, the time required for extracting picture position information and slice position information is shortened.

  The processing of the other steps is the same as the processing described above with reference to FIGS. 13 and 14, and the description thereof will be omitted because it will be repeated. Note that the decoding start position of the picture layer and the decoding start position of each decoder 365 are detected based on the clip position information.

  Note that, by combining the processing described above with reference to FIGS. 3 to 6 and the processing described above with reference to FIGS. 7 to 15 or FIGS. When the division decoding area is set so as to include the picture position information and the slice position information is recorded, the decoding of the video stream can be further speeded up. Hereinafter, an embodiment in which the processing described above with reference to FIGS. 3 to 6 and the processing described above with reference to FIGS. 7 to 15 are combined will be described. Note that the encoding of the video stream is the same as the processing described above with reference to FIGS. 7, 11, and 12, and the description thereof will be omitted because it will be repeated.

  FIG. 24 is a diagram illustrating an example of a functional configuration of the decoding unit 451 realized by the CPU 121 that executes a predetermined program. The decoding unit 451 is different from the decoding unit 251 in FIG. 8 in that a decoding area setting unit 463 is provided. In the figure, portions corresponding to those in FIG. 8 are denoted by the same reference numerals in the last two digits, and the description of the portions having the same processing will be omitted because it will be repeated.

  The decoding unit 451 includes a stream read control unit 461, a stream analysis unit 462, a decode area setting unit 463, a decode control unit 464, decoders 465-1 to 465-4, a baseband output control unit 466, and a start position detection unit 471. It is comprised so that it may contain.

  The stream analysis unit 462 decodes layers up to the picture layer of the video stream, and supplies information obtained by decoding to the decode control unit 464. In addition, the stream analysis unit 462 supplies information indicating the image size of the video stream obtained by decoding the sequence layer of the video stream to the decoding region setting unit 463. Further, the stream analysis unit 462 extracts picture position information and slice position information recorded in the video stream, and supplies the extracted picture position information and slice position information to the start position detection unit 471.

  The decode area setting unit 463 sets a divided decode area in the same manner as the decode area setting unit 163 in FIG. The decode area setting unit 163 supplies information indicating the set divided decode area to the decode control unit 464.

  In addition to the processing of the decode control unit 264 in FIG. 8, the decode control unit 464 determines slices to be decoded by the decoders 465-1 to 465-4 based on the divided decode region.

  Hereinafter, the decoders 465-1 to 465-4 are simply referred to as a decoder 465 when it is not necessary to distinguish them individually.

  Next, the decoding process executed by the decoding unit 451 will be described with reference to the flowcharts of FIGS.

  In step S351, it is determined whether or not the entire video stream has been decoded by the same processing as in step S1 of FIG. If it is determined that the video stream has not been completely decoded, the process proceeds to step S352.

  In step S352, the sequence layer of the video stream to be decoded is read from the removable medium 116 by the same process as in step S2 of FIG. 4 described above, and the sequence layer is decoded.

  In step S353, the image size of the video stream is detected by the same processing as in step S3 of FIG. 4 described above.

  In step S354, a divided decoding area to be decoded by each decoder 465 is set by the same processing as in step S4 in FIG. 4 described above, and information indicating the divided decoding area is supplied to the decoding control unit 464.

  In step S355, it is determined whether or not all GOPs have been decoded by the same processing as in step S5 of FIG. 4 described above. If it is determined that not all GOPs have been decoded yet, the process proceeds to step S356.

  In step S356, the GOP to be decoded next is read from the removable medium 116 by the same processing as in step S6 of FIG. 4 described above, and the GOP layer of the read GOP is decoded.

  In step S357, picture position information is extracted by the same processing as in step S155 of FIG. 13 described above, and the picture position information is supplied to the start position detection unit 471.

  In step S358, it is determined whether or not all the pictures in the GOP have been decoded by the same processing as in step S7 of FIG. 4 described above. If it is determined that all the pictures in the GOP have not been decoded yet, the process proceeds to step S359.

  In step S359, the picture layer of the picture to be decoded next is read out by the same processing as in step S157 of FIG. 13 described above, and the picture layer is supplied to the stream analysis unit 462.

  In step S360, the picture layer is decoded by the same process as in step S9 of FIG. 4 described above.

  In step S361, slice position information is acquired by the same processing as in step S159 of FIG. 13 described above, and the slice position information is supplied to the start position detection unit 471.

  In step S362, the start position detection unit 471 detects the start position of the divided decode area. Specifically, the decode control unit 464 detects the start position of each divided decode area by designating the slice number of the first slice of the divided decode area decoded by each decoder 465 in the next picture to be decoded. Is supplied to the start position detector 471. The start position detection unit 471 detects the start position of the slice of the designated slice number, that is, the start position of each divided decode area, based on the slice position information. The start position detection unit 471 supplies information indicating the detected start position to the decode control unit 464.

  In step S363, the decode control unit 464 instructs the start of decoding. Specifically, the decode control unit 464 supplies the stream read control unit 461 with information instructing the supply to each decoder 465 of the slices included in each divided decode region starting from the start position detected in step S362. In addition, the decoding control unit 464 supplies information indicating the start of decoding of the next picture to each decoder 465.

  In step S364, the decoder 465 starts decoding. Specifically, the stream read control unit 461 reads the slice included in each divided decode area starting from the start position instructed by the decode control unit 464 from the stream memory 152. The stream read control unit 461 supplies the read slice to the decoder 465 to which the decoding of the divided decoding area is assigned for each divided decoding area. The decoder 465 starts decoding from the first slice in the divided decoding area. The decoder 465 sequentially supplies the decoded data to the baseband output control unit 466. The baseband output control unit 466 stores the acquired data in the baseband memory 153.

  In step S365, whether or not an error has occurred is determined by the same processing as in step S13 of FIG. If it is determined that an error has occurred, the process proceeds to step S366.

  In step S366, the decode control unit 464 determines a position at which decoding is resumed. For example, the decode control unit 464 determines a slice next to a slice that has failed to be decoded as a position to resume decoding.

  In step S367, the decoder 465 resumes decoding. Specifically, the decode control unit 464 supplies information instructing to resume decoding from the slice following the slice in which the error has occurred to the decoder 465 in which the error has occurred. The decoder 465 resumes decoding from the designated slice.

  If it is determined in step S365 that no error has occurred, the processes in steps S366 and S367 are skipped, and the process proceeds to step S368.

  In step S368, it is determined whether or not the processing of all the decoders 465 has been completed by the same processing as in step S16 of FIG. If it is determined that the processes of all the decoders 465 have not been completed yet, the process returns to step S365, and in steps S368, the processes of steps S365 to S368 are performed until it is determined that the processes of all the decoders 465 have been completed. The process is executed repeatedly.

  If it is determined in step S368 that the processing of all the decoders 465 has been completed, the processing proceeds to step S369.

  In step S369, data for one frame is output by the same process as in step S17 of FIG. Thereafter, the process returns to step S358.

  If it is determined in step S358 that all the pictures in the GOP have not been decoded yet, the process proceeds to step S359, and the processes after step S359 described above are executed. That is, the next picture is decoded.

  If it is determined in step S358 that all the pictures in the GOP have been decoded, the process returns to step S355.

  If it is determined in step S355 that not all GOPs have been decoded yet, the process proceeds to step S356, and the processes after step S356 described above are executed. That is, the next GOP is decoded.

  If it is determined in step S355 that all GOPs have been decoded, the process returns to step S351.

  If it is determined in step S351 that the entire video stream has not been decoded yet, the process proceeds to step S352, and the processes after step S352 described above are executed. That is, the next sequence layer of the video stream is decoded.

  If it is determined in step S351 that the entire video stream has been decoded, the decoding process ends.

  In this way, when the stream analysis unit 462 decodes the picture layer, the decoding start position can be quickly detected based on the picture position information without searching for the start position (start code) of the picture layer on the video stream. Is done. Further, when each decoder 465 decodes, it is only necessary to detect the start position of each divided decode area as the position at which each decoder 465 starts decoding for each frame (picture), and the slice layer on the video stream. Without searching for the start position (start code), the start position of each divided decode area is quickly detected based on the slice position information. Therefore, the time for detecting the position to start decoding is shortened, and the decoding of the video stream can be speeded up. Similarly, when decoding is performed by one decoder without dividing the decoding process, the time for detecting the position to start decoding is shortened, and the decoding of the video stream can be speeded up.

  Note that also when the process described above with reference to FIGS. 3 to 6 and the process described above with reference to FIGS. 16 to 23 are combined, the picture position information and the slice position information are recorded in the clip position information. Except for the above, the processing is almost the same as the processing described above with reference to FIGS.

  Further, it is possible to further speed up the decoding process by further dividing the decoding process of the video stream into a plurality of stages and performing the divided decoding process in each stage by a plurality of hardware in parallel. With reference to FIGS. 27 to 37, an embodiment will be described in which the decoding process is divided into a plurality of stages and the divided decoding processes are performed in parallel by a plurality of hardware.

  FIG. 27 is a diagram showing an embodiment of an AV processing system 501 in which the decoding process is divided into a plurality of stages and the divided decoding processes are performed in parallel by a plurality of hardware. In the figure, portions corresponding to those in FIG. 2 are denoted by the same reference numerals, and description of portions having the same processing will be omitted because it will be repeated.

  Compared with the AV processing system 101 of FIG. 2, the AV processing system 501 is common in that it includes a drive 112, external video recording / playback devices 113-1 to 113-n, a mouse 114, and a keyboard 115. The difference is that an AV processing device 511 is provided instead of the device 111.

  Further, the AV processing device 511 has a CPU 121, a ROM 122, a RAM 123, a hard disk 124, a signal processing unit 125, a video special effect audio mixing processing unit 126, a display 127, a speaker 128, and the AV processing device 111 of FIG. The difference is that the interfaces 129-1 to 129-3 are included, and a GPU (Graphics Processing Unit) 521 is provided. Note that the CPU 121, ROM 122, RAM 123, HDD 124, signal processing unit 125, video special effect audio mixing processing unit 126, interfaces 129-1 to 129-3, and GPU 521 are connected to each other via the bus 130.

  The GPU 521 is a processor that mainly performs graphics processing. In the AV processing device 511, as will be described later, the two hardware (processors) of the CPU 121 and the GPU 521 divide the video stream decoding process into two stages and execute the divided decoding processes in parallel. To do.

  FIG. 28 is a diagram illustrating an example of a functional configuration of the decoding unit 551 realized by the CPU 121 that executes a predetermined program. In the figure, portions corresponding to those in FIG. 24 are denoted by the same reference numerals in the last two digits, and description of portions having the same processing will be omitted because it will be repeated.

  The decode unit 551 includes a stream read control unit 561, a stream analysis unit 562, a decode area setting unit 563, a decode control unit 564, decoders 565-1 to 565-4, a start position detection unit 571, a slice data storage memory 572, and a transfer The memory 573 and the memory transfer control unit 574 are configured to be included.

  The stream read control unit 561 controls the drive 112 to read the video stream recorded on the removable medium 116. The stream read control unit 561 acquires the video stream read by the drive 112 via the interface 129-3 and the bus 130. Further, the stream read control unit 561 reads a video stream recorded in the HDD 124 via the bus 130. The stream read control unit 561 temporarily stores the acquired video stream in the stream memory 152 including a cache memory (not shown) in the CPU 121 as necessary. In addition, the stream read control unit 561 supplies the acquired video stream to the stream analysis unit 562 or the decoders 565-1 to 565-4 as necessary.

  In addition to the processing of the decoding control unit 464 in FIG. 24, the decoding control unit 564 executes decoding processing up to a predetermined middle stage of the slice in the divided decoding area (decoding first half process described later with reference to FIG. 32). The decoders 565-1 to 565-4 are controlled so as to execute the first half decoding process on the divided decoding area assigned to each decoder in parallel with the other decoders. Further, the decode control unit 564 executes the decoding process (decoding latter half process described later with reference to FIG. 33 and FIG. 34) of the remaining slice of the slice decoded to the middle stage by the decoders 565-1 to 565-4. And the memory transfer control unit 574 and the GPU 521 are controlled to execute the latter half of the decoding process in parallel with the first half decoding process by the decoders 565-1 to 565-4.

  Further, the decode control unit 564 acquires information notifying that the processing of the GPU 521 has been completed from the GPU 521 via the bus 130. Also, the decode control unit 564 supplies information indicating whether or not the GPU 521 is executing processing to the decoders 565-1 to 565-4. Further, the decode control unit 564 supplies information for instructing the start of the first half of the decoding process to the decoders 565-1 to 565-4 (calls a thread for executing the first half decoding process and is necessary for the first half decoding process to the calling thread. Give information).

  Further, the decode control unit 564 supplies information for instructing the start of the second half of the decoding process to the memory transfer control unit 574 and the GPU 521 (calls a thread for executing the second half decoding process, and the information necessary for the second half decoding process is called to the calling thread) give). The information instructing the start of the latter half of the decoding process includes, for example, history information indicating how far the slice to be decoded has been decoded.

  The decoders 565-1 to 565-4 decode (variable length decoding and inverse quantization) the divided decoding areas in each frame of the video stream assigned to each decoder in parallel. Decoders 565-1 to 565-4 are configured to include one set each of VLD (variable length decoding) units 581-1 to 581-4 and IQ (inverse quantization) units 582-1 to 582-4. . Hereinafter, when it is not necessary to individually distinguish the decoders 565-1 to 565-4, they are simply referred to as the decoder 565, and when it is not necessary to individually distinguish the VLD units 581-1 to 581-4, they are simply referred to as the VLD unit. When it is called 581 and it is not necessary to individually distinguish the IQ parts 582-1 to 582-4, they are simply called IQ parts 582.

  The VLD unit 581 performs variable length decoding on the data (video stream) supplied from the stream read control unit 561 and supplies the variable length decoded data to the IQ unit 582. In addition, the VLD unit 581 supplies information indicating the prediction mode, the motion vector, and the frame / field prediction flag included in the decoded data to the GPU 521 via the bus 130. Further, the VLD unit 581 supplies information indicating the quantization scale included in the decoded data to the IQ unit 582.

  The IQ unit 582 dequantizes the data supplied from the VLD unit 581 according to the quantization scale supplied from the VLD unit 581 and stores the dequantized data in the slice data storage memory 572. Further, when the data that has been inversely quantized by the IQ unit 582 is stored in the slice data storage memory 572 for one slice, the IQ unit 582 transfers the data for one slice from the slice data storage memory 572 to the transfer memory 573.

  The memory transfer control unit 574 transfers the data stored in the transfer memory 573 to the GPU 521 via the bus 130.

  FIG. 29 is a diagram illustrating an example of a functional configuration of the decoding unit 601 realized by the GPU 521 that executes a predetermined program. The decoding unit 601 is configured to include a slice data memory 611, an IDCT (Inverse Discrete Cosine Transform) unit 612, a motion compensation unit 613, a frame data generation unit 614, and a frame memory 615.

  The slice data memory 611 stores the inversely quantized data for one slice supplied from the memory transfer control unit 574 via the bus 130.

  The IDCT unit 612 performs an IDCT process, which will be described later with reference to FIG. The IDCT unit 612 supplies the image data obtained by performing the IDCT process to the frame data generation unit 614. Note that the IDCT unit 612 performs IDCT of the macroblock of the video stream, for example, based on the Fast IDCT algorithm. For details on the Fast IDCT algorithm, see, for example, “Yukihiro Arai and 2 others, Urine Fast DCT-SQ Scheme for Images THE TRANSACTIONS OF THE IEICE, VOL.E 71, NO.11 NOBEMBER 1988, pp.1095-1097” The details are disclosed.

  The motion compensation unit 613 acquires information indicating the prediction mode, the motion vector, and the frame / field prediction flag supplied from the VLD unit 581 via the bus 130. When the next image data generated by the frame data generation unit 614 is a P picture in the forward prediction mode, the motion compensation unit 613 stores the image data of the previous frame stored in the past reference image unit 615a of the frame memory 615. Then, motion compensation corresponding to the motion vector supplied from the VLD unit 581 is performed to generate predicted image data. The motion compensation unit 613 supplies the generated predicted image data to the frame data generation unit 614.

  In addition, when the image data generated next by the frame data generation unit 614 is a B picture, the motion compensation unit 613 corresponds to the prediction mode supplied from the VLD unit 581 and the past reference image unit 615a of the frame memory 615. Stored in the image data (in the case of the forward prediction mode), the image data stored in the future reference image unit 615b (in the case of the backward prediction mode), or both of the image data (in the bidirectional prediction mode). In the case), motion compensation corresponding to the motion vector supplied from the VLD unit 581 is performed to generate predicted image data. The motion compensation unit 613 supplies the generated predicted image data to the frame data generation unit 614.

  Further, when the image data stored in the future image reference unit 615b is a P picture, the motion compensation unit 613 determines that the future image is decoded when all the decoding of the B picture located in time series before the P picture is completed. The P picture stored in the reference unit 615b is read and supplied to the frame data generation unit 614 without performing motion compensation.

  The frame data generation unit 614 stores the image data supplied from the IDCT unit 612 under the control of the decode control unit 564, generates image data for one frame, and outputs the generated image data. Specifically, when the image data supplied from the IDCT unit 612 is an I picture, or a P picture or B picture in the intra-frame prediction mode, the frame data generation unit 614 supplies the image data supplied from the IDCT unit 612. One frame of image data generated based on the above is generated. In addition, when the image data supplied from the IDCT unit 612 is a P picture or B picture that is not in the intra-frame prediction mode, the frame data generation unit 614 generates image data generated based on the image data supplied from the IDCT unit 612. The predicted image data supplied from the motion compensation unit 613 is added to generate image data for one frame.

  When the generated image data is an I picture, the frame data generation unit 614 outputs the generated image data to the outside of the decoding unit 601 (for example, the signal processing unit 125 via the bus 130) and the frame memory 615. Are stored in the past reference image portion 615a or the future reference image portion 615b. In addition, when the generated image data is a P picture, the frame data generation unit 614 stores the generated image data in the future reference image unit 615b of the frame memory 615 without outputting the generated image data to the outside of the decoding unit 601. Furthermore, when the generated image data is a B picture, the frame data generation unit 614 outputs the generated image data to the outside of the decoding unit 601.

  Further, when the image data (P picture) stored in the future image reference unit 615 b is supplied from the motion compensation unit 613, the frame data generation unit 614 outputs the supplied image data to the outside of the decoding unit 601. .

  Furthermore, the frame data generation unit 614 supplies information notifying that the latter half of the decoding process has been completed to the decode control unit 564 via the bus 130.

  As described above, the frame memory 615 stores the image data (I picture or P picture) used to generate the predicted image data in the past reference image unit 615a or the future reference image unit 615b. Further, image data is transferred (bank switching) between the past reference image unit 615a and the future reference image unit 615b as necessary.

  Next, decoding processing executed by the decoding unit 551 will be described with reference to the flowcharts of FIGS. Hereinafter, in order to simplify the description, it is assumed that the block type of each macroblock is an intra (intraframe coding) type. Further, for the sake of simplicity, the steps related to the error occurrence processing described above (for example, steps S365 to S367 in FIG. 26) are omitted in the flowcharts in FIGS. 30 and 31.

  The processing of step S451 and step S462 is the same as the processing of steps S351 to S362 of FIG. 25 and FIG. 26 described above, and the description thereof will be omitted because it will be repeated.

  In step S463, the decode control unit 564 instructs the start of the first half of the decoding process. Specifically, the decode control unit 564 supplies the stream read control unit 461 with information instructing the supply to each decoder 465 of the slice included in each divided decode area starting from the start position detected in step S462. In addition, the decode control unit 464 supplies information indicating the start of the first half of the decoding process for the next picture to each decoder 565.

  In step S464, each decoder 565 performs the first half of the decoding process. Details of the first decoding process will be described later with reference to FIG.

  In step S465, the decode control unit 564 determines whether or not the processing of all the decoders 565 has been completed. Specifically, the decode control unit 564 detects whether or not information notifying the end of decoding of all slices in the divided decoding area allocated to each decoder 565 is supplied from all the decoders 565. The decode control unit 564 repeats the determination process in step S465 until information for notifying the end of decoding is supplied from all the decoders. When the information for notifying the end of decoding is supplied from all the decoders, Is determined to have been completed, and the process returns to step S458. Thereafter, the processing after step S458 is executed.

  Next, details of the first half of the decoding process in step S464 in FIG. 31 will be described with reference to the flowchart in FIG.

  In step S501, the VLD unit 581 performs variable length decoding. Specifically, the VLD unit 581 performs variable length decoding on the first macroblock of the slice supplied from the stream read control unit 561. The VLD unit 581 supplies the variable length decoded macroblock to the IQ unit 582. Also, the VLD unit 581 supplies information indicating the quantization scale used for inverse quantization of the macroblock to the IQ unit 582.

  In step S502, the IQ unit 582 performs inverse quantization. Specifically, the IQ unit 582 performs inverse quantization on the macroblock supplied from the VLD unit 581 according to the quantization scale supplied from the VLD unit 581.

  In step S503, the IQ unit 582 stores the data dequantized in step S502 in the slice data storage memory 572.

  In step S504, the IQ unit 582 determines whether data for one slice has been accumulated. Specifically, the IQ unit 582 determines that the data for one slice is not accumulated when the data that has been inversely quantized by the IQ unit 582 has not reached one slice. Then, the process returns to step S501, and the processes of steps S501 to S504 are repeatedly executed until it is determined in step S504 that data for one slice has been accumulated. That is, the second and subsequent macroblocks from the beginning of the slice are sequentially subjected to variable length decoding and inverse quantization.

  In step S504, the IQ unit 582 determines that the data for one slice is stored when the data dequantized by itself stored in the slice data storage memory 572 reaches one slice. The process proceeds to step S505.

  In step S505, the IQ unit 582 determines whether the processing of the GPU 521 has ended. Specifically, the IQ unit 582 acquires information indicating whether or not the GPU 521 is executing processing from the decode control unit 564. If the GPU 521 is executing the IDCT process, the IQ unit 582 determines that the process of the GPU 521 has not ended, and the process proceeds to step S506.

  In step S506, the IQ unit 582 determines whether all macroblocks in the divided decoding area have been processed. If the IQ block 582 has not yet processed all the macroblocks in the divided decoding area, that is, if the macroblock that has not yet been variable-length decoded and dequantized is in the divided decoding area assigned to itself, Returning to step S501, variable-length decoding and inverse quantization of the macroblock in the divided decoding area are continuously performed.

  If it is determined in step S506 that all the macroblocks in the divided decoding area have been processed, that is, all the macroblocks in the divided decoding area assigned to the variable decoding and inverse quantization have been performed. The process proceeds to step S507.

  In step S507, as in the process of step S505, it is determined whether or not the processing of the GPU 521 has ended. Until it is determined that the processing of the GPU 521 has ended, the processing of step S507 is repeatedly executed. When it is determined that the processing of the GPU 521 has ended, that is, when the GPU 521 is in the idle state, The process proceeds to S508.

  In step S508, the IQ unit 582 transfers the dequantized data. Specifically, the IQ unit 582 transfers the data of the first slice of the data stored in the slice data storage memory 572 and dequantized by itself to the transfer memory 573.

  In step S509, the decode control unit 564 instructs the start of the second half decoding process. The decode control unit 564 supplies the memory transfer control unit 574 and the decoding unit 601 with information instructing the start of the latter half of the decoding process for the slice accumulated in the transfer memory 573. Thereby, the latter half of the decoding process described later with reference to FIG. 33 is started. Thereafter, the process proceeds to step S513.

  If it is determined in step S505 that the processing of the GPU 521 has been completed, the processing proceeds to step S510.

  In step S510, similarly to the processing in step S507, the inversely quantized data for one slice is transferred to the transfer memory 573.

  In step S511, the start of the latter half of the decoding process is instructed in the same manner as the process in step S509 described above.

  In step S512, it is determined whether or not all macroblocks in the divided decoding area have been processed, similar to the process in step S506. If it is determined that all the macroblocks in the divided decoding area have not yet been processed, the process returns to step S501, and variable length decoding and inverse quantization of the macroblock in the divided decoding area are continued.

  If it is determined in step S512 that all macroblocks in the divided decoding area have been processed, the process proceeds to step S513.

  In step S513, the IQ unit 582 determines whether all the dequantized data has been transferred. If the data that has not been transferred to the transfer memory 573 among the data that has been dequantized by the IQ unit 582 remains in the slice data storage memory 572, the IQ unit 582 has not yet transferred all the dequantized data. Determination is made, and the process returns to step S507. Thereafter, the processes in steps S507, S508, and S513 are repeatedly executed until it is determined in step S513 that all the dequantized data has been transferred.

  If it is determined in step S513 that all the dequantized data has been transferred, the process proceeds to step S514.

  In step S514, the decoder 565 notifies the end of the first half of the decoding process, and the first half of the decoding process ends. Specifically, the decoder 565 supplies information that notifies the end of the first half of the decoding process to the decoding control unit 564.

  In the above description, the first half of the decoding process has been described with a focus on one decoder. In practice, this first half of the decoding process is executed in parallel by the decoders 565-1 to 565-4.

  Next, the decoding latter half process executed by the decoding unit 551 and the decoding unit 601 corresponding to the decoding first half process of FIG. 32 will be described with reference to the flowchart of FIG. This process is executed in parallel with the first half decoding process executed by each decoder 565 described above with reference to FIG.

  In step S521, the memory transfer control unit 574 determines whether or not an instruction to start the latter half of the decoding process has been issued. Until it is determined that the start of the latter half of the decoding process has been instructed, the determination process of step S521 is repeatedly executed. In step S509 or S511 of FIG. When supplied to the memory transfer control unit 574, it is determined that the start of the second half of the decoding process has been instructed, and the process proceeds to step S522.

  In step S522, the memory transfer control unit 574 transfers data. Specifically, the memory transfer control unit 574 transfers the inversely quantized data for one slice stored in the transfer memory 573 to the slice data memory 611 of the decoding unit 601 via the bus 130. , Remember.

  In step S523, the IDCT unit 612 performs IDCT (inverse discrete cosine transform) processing. Details of the IDCT process will be described later with reference to FIG.

  In step S524, the frame data generation unit 614 determines whether image data for one frame has been decoded. If it is determined that the image data for one frame has not been decoded, the process proceeds to step S525.

  In step S525, the frame data generation unit 614 outputs image data for one frame. Specifically, the frame data generation unit 614 outputs image data for one frame for which decoding has been completed to the outside (for example, the signal processing unit 125 via the bus 130).

  If it is determined in step S524 that the image data for one frame has not been decoded, the process in step S525 is skipped, and the process proceeds to step S526.

  In step S526, the frame data generation unit 614 notifies the end of the second half of the decoding process. Specifically, the frame data generation unit 614 supplies information notifying the end of the second half of the decoding process to the decode control unit 564 via the bus 130. Thereafter, the process returns to step S521, and the processes after step S521 described above are executed.

  Next, details of the IDCT processing in step S523 in FIG. 33 will be described with reference to the flowchart in FIG.

  In step S541, the IDCT unit 612 generates a texture. Specifically, the IDCT unit 612 reads data for one slice stored in the slice data memory 611.

  In the following description, it is assumed that the data for one slice is composed of the format shown in the slice 651 in FIG. The slice 651 is composed of n macro blocks (MB). One macroblock includes 16 × 16 pieces of luminance information Y and 16 × 8 pieces of color difference information Cb and Cr, respectively. That is, the ratio of the three components of the luminance information Y and the color difference information Cb and Cr is 4: 2: 2. Also, 16 × 16 luminance information Y, 16 × 8 chrominance information Cb, 16 × 8 chrominance information Cr, 16 × 8 chrominance information Cb, 16 × vertical Each piece of information is arranged from the top of the macroblock in the order of eight pieces of color difference information Cr.

  For each macroblock included in the slice 651, the IDCT unit 612 has luminance information Y × 64, color difference information Cb × 16, color difference information Cr × 16, color difference information Cb × 16, color difference information per macroblock. Four columns of information arranged in the order of Cr × 16 are generated. The IDCT unit 612 generates four textures 661 to 664 in which four columns of information generated from each macroblock are arranged in order for each column. Therefore, one texture is composed of vertical n × horizontal 128 pieces of information.

  In step S542, the IDCT unit 612 performs row conversion. Specifically, the IDCT unit 612 uses the information arranged in the top row (horizontal column) of the textures 661 to 664, that is, the information included in the top macroblock of the slice 651. A three-dimensional block 671-1 composed of 8 horizontal × 8 deep information is generated.

  Each piece of information constituting the three-dimensional block 671-1 is obtained by multiplying each piece of information in the first row (horizontal column) of the textures 661 to 664 by a predetermined coefficient and multiplying the information according to a predetermined rule. Generated by adding together. The luminance information Y is arranged in the first to fourth lines from the top of the block 671-1, and the color difference information Cb is arranged in the fifth and sixth lines from the top, and the seventh and eighth lines from the top. Color difference information Cr is arranged in the eye.

  In step S543, the IDCT unit 612 determines whether all the row conversions have been completed. If it is determined that the row conversion has not been completed yet, the process returns to step S542, and the processes of steps S542 and S543 are repeatedly executed until it is determined in step S543 that all the row conversions have been completed. That is, the row conversion described above is sequentially performed on each row of the textures 661 to 664, and as a result, three-dimensional blocks 671-1 to 671-n are generated.

In the following description, eight two-dimensional blocks comprising eight pieces of vertical information and eight pieces of horizontal information constituting the three-dimensional block 671-m (m = 1, 2, 3,... N) are represented by the two-dimensional block 671-m. _{1 to} 671-m ₈ (m = 1, 2, 3,... N).

  If it is determined in step S543 that all the row conversions have been completed, the process proceeds to step S544.

  In step S544, the IDCT unit 612 performs column conversion. Specifically, the IDCT unit 612 is composed of 3 pieces of information of 8 vertical × 8 horizontal × 8 depth from the three-dimensional block 671-1 generated from the top macroblock of the slice 651 according to a predetermined rule. A dimension block 681-1 is generated.

The information of each column in the depth direction of the three-dimensional block 681-1 may be obtained by multiplying the information of the row in the horizontal direction of the three-dimensional block 671-1 associated with each column by a predetermined rule by a predetermined coefficient. , And the information multiplied by the coefficient are added together. For example, three-dimensional block uppermost Diagram depth direction information of the leftmost column of the surface of the 681-1 is two-dimensional blocks 671-1 ₁ of FIG constituting the three-dimensional block 671-1 that corresponds to that column It is generated by multiplying the information in the first row in the horizontal direction by a predetermined coefficient or adding the information multiplied by the coefficient. The luminance information Y is arranged in the first to fourth lines from the top of the block 681-1, the color difference information Cb is arranged in the fifth and sixth lines from the top, and the seventh and eighth lines from the top. Color difference information Cr is arranged in the eye.

  In step S545, the IDCT unit 612 determines whether all column conversions have been completed. If it is determined that all the column conversions have not been completed, the process returns to step S544, and the processes of steps S544 and S545 are repeatedly executed until it is determined in step S545 that all the column conversions have been completed. That is, the above-described column conversion is sequentially performed on the three-dimensional blocks 671-1 to 671-n, and as a result, three-dimensional blocks 681-1 to 681-n are generated.

In the following description, eight two-dimensional blocks comprising eight pieces of vertical information and eight pieces of horizontal information constituting the three-dimensional block 681-m (m = 1, 2, 3,... N) are represented by the two-dimensional block 681-m. _{1 to} 681-m ₈ (m = 1, 2, 3,... N).

  If it is determined in step S545 that all column conversions have been completed, the process proceeds to step S546.

In step S546, the IDCT unit 612 rearranges the information. Specifically, IDCT unit 612, as shown in FIG. 36 is included at the top side of the three-dimensional block 681-1 (topmost row of the two-dimensional blocks 681-1 ₁ to 681-1 ₈₎ An 8 × 8 macroblock 711-1 is generated from the luminance information Y. Similarly, the IDCT unit 612 generates a macroblock 711-2 from the luminance information Y included in the second surface from the top of the block 681-1, and the macroblock from the luminance information Y included in the third surface from the top. 711-3 is generated, and a macroblock 711-4 is generated from the luminance information Y included in the fourth surface from the top.

  Further, when the picture including the slice being decoded has a field structure, the IDCT unit 612 generates macro blocks 712-1 and 712-2 in which the rows of the macro blocks 711-1 and 711-3 are alternately arranged, Macro blocks 712-3 and 712-4 in which the rows of the macro blocks 711-2 and 711-4 are alternately arranged are generated. In the figure, the IDCT unit 612 is a 16 × 16 macro block 713 in which the macro block 712-1 is arranged at the upper left, the macro block 712-2 is arranged at the lower left, the macro block 712-3 is arranged at the upper right, and the macro block 712-4 is arranged at the lower right. Is generated.

  When the picture including the slice being decoded has a frame structure, the IDCT unit 612 has the macro block 711-1 in the upper left, the macro block 711-2 in the upper right, the macro block 711-3 in the lower left, and the macro block. A 16 × 16 macroblock 714 in which 711-4 is arranged in the lower right is generated.

  The IDCT unit 612 performs similar rearrangement on the luminance information Y included in the three-dimensional blocks 681-2 to 681-n. Similar rearrangement is performed on the color difference information Cr and Cb included in the three-dimensional blocks 681-1 to 681-n. In addition, the IDCT unit 612 generates a slice 691 in which macroblocks in which information is rearranged are arranged in order from the top. For example, when a picture including a slice being decoded has a field structure, as shown in FIG. 37, macroblocks 712-1 to 712-4 generated from the three-dimensional block 681-1 are the top of the slice 691 (see FIG. In the same manner, the macroblocks generated from the three-dimensional blocks 681-2 to 688-n are sequentially arranged.

  In step S547, the IDCT unit 612 performs RGB conversion, and the IDCT processing ends. Specifically, the image data 701 is generated by converting the luminance information Y and the color difference information Cb and Cr constituting the slice 691 into RGB R, G, and B signals. The IDCT unit 612 supplies the generated image data 701 to the frame generation unit 614. In addition, the IDCT unit 612 supplies information notifying that the IDCT processing is completed to the decode control unit 564 via the bus 130.

  As described above, the decoding processing below the slice layer is divided and executed in parallel by the CPU 121 and the GPU 521, whereby the time required for the decoding processing can be reduced and the load on the CPU 121 can be reduced. In addition, the cost can be reduced by using an inexpensive GPU 521 as compared with a high-performance processor such as a CPU.

  As described above, the areas corresponding to the pictures of the video stream are divided areas according to the number of the first decoding means, and the processing of the first decoding means and the second decoding means for decoding the video stream A division area including a plurality of unit areas as a unit is set, and decoding up to a predetermined intermediate stage of the unit area in the division area assigned to each of the first decoding means is performed in parallel with the other first decoding means. The second decoding means controls the first decoding means so as to execute the decoding, and executes the decoding of the remaining stage of the unit area decoded up to an intermediate stage in parallel with the decoding by the first decoding means. When controlling the video stream, the video stream can be decoded at a higher speed.

  In the above description, an example in which the number of decoders realized by the CPU is four is shown, but a number other than four may be used.

  In the above description, the example in which the number of decoders realized by the GPU is one has been described. However, a plurality of decoders may be realized by one GPU, or a plurality of decoders may be provided. The latter half of the decoding process may be executed in parallel by a plurality of decoders.

  Furthermore, in the above description, the CPU 121 is configured by a multi-core processor. However, a plurality of processors such as CPUs may be provided, and the processing of each decoder realized by the CPU 121 may be performed in parallel by a plurality of processors. Good.

  In the above description, an example in which a video stream is encoded or decoded by the MPEG2 system has been described. However, the present invention can be applied to, for example, the position of a frame of a video stream (for example, the relative position from the beginning of the video stream, recording Video by a method that does not record information indicating the position of the recording unit on the medium or the position of the processing unit when decoding the video stream (for example, the relative position from the beginning of the video stream, the position of recording on the recording medium, etc.) This can be applied when encoding or decoding a stream. In addition, the present invention provides, for example, a case where a video stream encoded by a method in which a plurality of processing units are included in an area on a video stream corresponding to an image of one frame of the video stream is decoded in parallel by a plurality of decoders. Can be applied.

  Further, in step S509 or S511 of FIG. 30, the decode control unit 564 stores history information indicating how far the slice stored in the transfer memory 573 is decoded in the transfer memory 573 in association with the slice. You may make it memorize | store. As a result, it becomes clear to which stage the slice stored in the transfer memory 573 is decoded by each decoder. For example, when decoding is divided into three or more stages, the transfer memory 573 The stored slice can be used as information for determining which decoder will decode next.

  The series of processes described above can be executed by hardware (for example, the signal processing unit 125 in FIG. 2 or FIG. 27) or can be executed by software. When a series of processing is executed by software, a program constituting the software is installed from, for example, a network or a recording medium into the AV processing device 111 or the like.

  As shown in FIG. 2 or FIG. 27, this recording medium is not only composed of a removable medium 117 on which a program is recorded, which is distributed to provide a program to a user separately from the computer. Are provided to the user in a state of being preinstalled in the ROM 122 on which a program is recorded, the HDD 124, or the like.

  Further, in the present specification, the step of describing the program stored in the recording medium is not limited to the processing performed in time series according to the described order, but is not necessarily performed in time series. It also includes processes that are executed individually.

  Further, in the present specification, the term “system” means an overall apparatus constituted by a plurality of apparatuses and means.

It is a figure explaining the example of the division | segmentation of a process at the time of decoding a video stream in parallel by several conventional processors. 1 is a block diagram showing an embodiment of an AV processing system to which the present invention is applied. It is a block diagram which shows the example of a function structure of the decoding part implement | achieved by CPU of FIG. It is a flowchart explaining the decoding process performed by the decoding part of FIG. It is a flowchart explaining the decoding process performed by the decoding part of FIG. FIG. 4 is a diagram illustrating an example of processing division when a video stream is decoded in parallel by the decoding unit of FIG. 3. It is a block diagram which shows the example of a function structure of the encoding part implement | achieved by CPU of FIG. It is a block diagram which shows the other example of a structure of the function of the decoding part implement | achieved by CPU of FIG. It is a figure which shows the example of a structure of the data of picture position information. It is a figure which shows the example of a structure of the data of slice position information. It is a flowchart explaining the encoding process performed by the encoding part of FIG. 12 is a flowchart for explaining details of encoding processing for a picture layer and below in step S105 of FIG. 11. It is a flowchart explaining the decoding process performed by the decoding part of FIG. It is a flowchart explaining the decoding process performed by the decoding part of FIG. It is a flowchart explaining the decoding start position detection process performed by the decoding part of FIG. It is a block diagram which shows the other example of a structure of the function of the encoding part implement | achieved by CPU of FIG. FIG. 10 is a block diagram showing still another example of a functional configuration of a decoding unit realized by the CPU of FIG. 2. It is a figure which shows the example of a structure of the data of clip position information. It is a figure which shows the example of a structure of the data of the picture position information contained in the clip position information of FIG. It is a figure which shows the example of a structure of the data of the slice position information contained in the clip position information of FIG. It is a flowchart explaining the encoding process performed by the encoding part of FIG. It is a flowchart explaining the decoding process performed by the decoding part of FIG. It is a flowchart explaining the decoding process performed by the decoding part of FIG. FIG. 10 is a block diagram showing still another example of a functional configuration of a decoding unit realized by the CPU of FIG. 2. It is a flowchart explaining the decoding process performed by the decoding part of FIG. It is a flowchart explaining the decoding process performed by the decoding part of FIG. It is a block diagram which shows other embodiment of the AV processing system to which this invention is applied. It is a block diagram which shows the example of a function structure of the decoding part implement | achieved by CPU of FIG. It is a block diagram which shows the example of a function structure of the decoding part implement | achieved by GPU of FIG. It is a flowchart explaining the decoding process performed by the decoding part of FIG. It is a flowchart explaining the decoding process performed by the decoding part of FIG. It is a flowchart explaining the detail of the decoding first half process of step S464 of FIG. It is a flowchart explaining the decoding latter half process performed by the decoding part of FIG. 28, and the decoding part of FIG. It is a flowchart explaining the detail of the IDCT process of step S423 of FIG. It is a figure explaining the transition of the data produced | generated by IDCT process. It is a figure explaining rearrangement of the information which constitutes a macroblock. It is a figure for demonstrating the arrangement | sequence of the information in a slice.

Explanation of symbols

  101 AV processing system, 111 AV processing device, 112 drive, 116, 117 removable media, 121 CPU, 122 ROM, 124 HDD, 125 signal processing unit, 151 decoding unit, 161 stream read control unit, 162 stream analysis unit, 163 decoding Area setting unit, 164 decoding control unit, 165 decoder, 166 baseband output control unit, 201 encoding unit, 211 encoder, 212 picture count unit, 213 slice count unit, 214 start position memory, 215 start position recording control unit, 216 stream Output control unit, 251 decoding unit, 261 stream reading control unit, 262 stream analysis unit, 264 decoding control unit, 265 decoder, 266 Baseband output control unit, 271 start position detection unit, 301 encoding unit, 311 encoder, 312 picture count unit, 313 slice count unit, 314 start position memory, 315 start position recording control unit, 316 stream output control unit, 321 GOP counter , 351 decode unit, 361 stream read control unit, 362 stream analysis unit, 364 decode control unit, 365 decoder, 366 baseband output control unit, 371 start position detection unit, 451 decode unit, 461 stream read control unit, 462 stream analysis 463 decode area setting unit 464 decode control unit 465 decoder 466 baseband output control unit 471 start position detection unit 501 AV processing system, 511 AV processing device, 521 GPU, 551 decoding unit, 561 stream read control unit, 562 stream analysis unit, 563 decoding area setting unit, 564 decoding control unit, 565 decoder, 571 start position detection unit, 572 slice Data storage memory, 574 memory transfer control unit, 601 decoding unit, 612 IDCT unit, 613 motion compensation unit, 614 frame data generation unit, 651 slice, 671 3D block, 681 3D block, 691 slice, 701 image data

Claims (10)

A divided area obtained by dividing an area corresponding to a picture of a video stream in accordance with the number of first decoding means, which is a processing unit of the first decoding means and the second decoding means for decoding the video stream. A setting step for setting a divided area including a plurality of unit areas;
The first decoding means so as to execute decoding up to a predetermined intermediate stage of the unit area in the divided area respectively assigned to the first decoding means in parallel with the other first decoding means. A first half decoding control step for controlling
A second half decoding control step of controlling the second decoding means so as to execute decoding of the remaining stages of the unit area decoded up to the intermediate stage in parallel with decoding by the first decoding means. A program that causes a computer to execute processing. The program according to claim 1, wherein the video stream is a video stream encoded by a moving picture experts group (MPEG) method. In the first half decoding control step, the first decoding means is controlled to perform decoding including variable length decoding and inverse quantization of the slice;
The program according to claim 2, wherein in the second half decoding control step, the second decoding unit is controlled to perform decoding including inverse discrete cosine transform of the slice. The program according to claim 2, wherein the unit area is a slice. The program according to claim 1, wherein the first decoding unit and the second decoding unit are realized by different hardware. The program according to claim 4, wherein the second decoding unit is realized by a GPU (Graphics Processing Unit). The program according to claim 1, wherein in the latter half decoding control step, information indicating to which stage decoding of the unit area has been completed is supplied to the second decoding means.   A recording medium on which the program according to claim 1 is recorded. A divided area obtained by dividing an area corresponding to a picture of a video stream in accordance with the number of first decoding means, which is a processing unit of the first decoding means and the second decoding means for decoding the video stream. A setting step for setting a divided area including a plurality of unit areas;
The first decoding means so as to execute decoding up to a predetermined intermediate stage of the unit area in the divided area respectively assigned to the first decoding means in parallel with the other first decoding means. A first half decoding control step for controlling
A second half decoding control step of controlling the second decoding means so as to execute decoding of the remaining stages of the unit area decoded up to the intermediate stage in parallel with decoding by the first decoding means. Decryption control method. A divided area obtained by dividing an area corresponding to a picture of a video stream in accordance with the number of first decoding means, which is a processing unit of the first decoding means and the second decoding means for decoding the video stream. Setting means for setting a divided area including a plurality of unit areas;
The first decoding means so as to execute decoding up to a predetermined intermediate stage of the unit area in the divided area respectively assigned to the first decoding means in parallel with the other first decoding means. Decoding control means for controlling the second decoding means so as to execute decoding of the remaining stages of the unit area decoded up to the intermediate stage in parallel with decoding by the first decoding means And a decoding control device comprising:

Images