JP2018088600A - Image processing apparatus, image processing method, and program - Google Patents

Abstract

An image processing apparatus and an image processing method capable of obtaining a predetermined image and an image of a selected area satisfactorily. An acquisition means for acquiring an image, a setting means for setting a selection area for selecting at least a part of the image, and referring to pixel values of other areas based on the setting of the selection area by the setting means. In addition, the image forming apparatus includes a dividing unit that divides an image into tiles that are regions that can be decoded without any delay, and an encoding unit that encodes tiles corresponding to the selected region. [Selection] Figure 4

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program.

  As an image processing device that compresses and encodes moving image data, an imaging device, a portable communication device, and the like are known. Such an image processing apparatus acquires a moving image signal by the imaging unit, compresses and encodes the acquired moving image signal, and records the compressed moving image file on a recording medium. In general, a portable communication device is used with a screen in a vertically long state. For this reason, even if the image is horizontally long, there is a user who browses the image on a vertically long display screen. Under such circumstances, there is a user who takes a vertically long image, that is, a vertical image and uploads the image to an SNS (Social Network Service), assuming that the image is browsed by the portable communication device. On the other hand, when viewing an image with a stationary television receiver or the like, since the image is viewed on a horizontally long screen, it is desirable to take a horizontally long image, that is, a horizontally image. Accompanying such diversification of the reproduction environment, an image pickup apparatus that can acquire a vertically long image in a state where the image pickup unit is held horizontally is proposed (see Patent Document 1).

JP 2005-286540 A

  However, the imaging device described in Patent Literature 1 cannot acquire a vertically long image and a horizontally long image. Here, it is conceivable to generate a vertically long image from a horizontally long image. However, in this case, since re-encoding is involved, the vertically long image thus obtained becomes a deteriorated image.

  An object of the present invention is to provide an image processing apparatus and an image processing method capable of obtaining a predetermined image and an image of a selected area satisfactorily.

  According to an aspect of the present invention, an imaging unit that captures an image, a setting unit that sets a selection region for selecting at least a part of the image, and other settings based on the setting of the selection region by the setting unit A dividing unit that divides the image into tiles that can be decoded without referring to pixel values of the region, and an encoding unit that encodes the tile corresponding to the selected region. An image processing apparatus is provided.

  ADVANTAGE OF THE INVENTION According to this invention, the image processing apparatus and image processing method which can acquire a predetermined image and the image of a selection area favorably can be provided.

1 is an external view showing an image processing apparatus according to a first embodiment. 1 is a block diagram illustrating an image processing apparatus according to a first embodiment. It is a block diagram which shows the encoding process part with which the image processing apparatus by 1st Embodiment is equipped. It is a figure which shows the example of a display of a selection area | region. It is a figure which shows the example of a structure of a tile. It is a flowchart which shows operation | movement of the image processing apparatus by 1st Embodiment. It is a flowchart which shows the example of an encoding process. It is a flowchart which shows operation | movement of the image processing apparatus by 2nd Embodiment. It is a flowchart which shows the example of an encoding process.

  Embodiments of the present invention will be described below in detail with reference to the accompanying drawings, but the present invention is not limited to the following embodiments.

[First Embodiment]
The image processing apparatus and the image processing method according to the first embodiment will be described in detail with reference to the drawings. FIG. 1 is an external view showing the image processing apparatus according to the present embodiment. The image processing apparatus 100 according to the present embodiment causes a subject image formed on a pixel array (not shown) of the imaging unit 112 (see FIG. 2) by the photographing lens 111 (see FIG. 2) via the operation unit 104. Shooting is performed according to a shooting instruction given by the user. Further, the image processing apparatus 100 according to the present embodiment compresses the size of the image data by encoding the image acquired by the imaging unit 112, that is, the image data. The image data encoded by the image processing apparatus 100 may be recorded on, for example, a removable nonvolatile recording medium 120 (see FIG. 2), or transferred to the external device 127 using wireless communication or the like. Also good. In this case, examples of the external device 127 include a device that can record an image file. Further, the image processing apparatus 100 according to the present embodiment can display a captured image, a preview image before photographing, and the like on the display screen 125 of the display unit 116.

  The image processing apparatus 100 according to the present embodiment, that is, the image encoding apparatus, has a vertically long image, that is, a vertical image, regardless of the attitude (inclination) of the imaging unit (acquisition unit, imaging unit, imaging element, image sensor) 112. It has a vertical shooting mode (vertical image shooting mode) that always takes images. In the vertical photographing mode, at least a vertically long image is acquired regardless of whether the imaging unit 112 is held horizontally or vertically. In the image processing apparatus 100 according to the present embodiment, the preview images sequentially acquired by the imaging unit 112 are displayed on the display screen 125 of the display unit 116 in the vertical shooting mode. In the vertical shooting mode, a selection area 122 for selecting at least a part of an image acquired by the imaging unit 112 is displayed on the display screen 125 of the display unit 116. The selection area 122 is set to a vertically long rectangle, for example, so that a desired vertically long image can be obtained. The selection area 122 can be changed as will be described later. More specifically, the selection area 122 can be moved, enlarged, reduced, etc., as will be described later. In the vertical shooting mode, the image processing apparatus 100 according to the present embodiment can simultaneously acquire not only a vertically long image, that is, a vertical image but also a horizontally long image, that is, a horizontal image.

  The body (housing, main body) 123 of the image processing apparatus 100 according to the present embodiment includes an operation unit 104 including various operation buttons. By appropriately operating these operation buttons, the user can perform an instruction to shoot a moving image, an instruction to shoot a still image, change a shooting mode, display a shot image, and the like. Here, the case where the image processing apparatus 100 according to the present embodiment is an imaging apparatus, specifically, a digital still camera will be described as an example, but the present invention is not limited thereto. For example, the image processing apparatus 100 may be a smartphone, a mobile phone with a camera, a digital video camera, or the like.

  FIG. 2 is a block diagram illustrating the image processing apparatus 100 according to the present embodiment. As shown in FIG. 2, the image processing apparatus 100 according to the present embodiment includes a CPU 101, a memory 102, a nonvolatile memory 103, an operation unit 104, an imaging unit 112, an image processing unit 113, and an encoding processing unit. 114, a display control unit 115, and a display unit 116. Furthermore, the image processing apparatus 100 according to the present embodiment includes a communication control unit 117, a communication unit 118, a recording medium control unit 119, a tilt detection unit 121, and an internal bus 130. The image processing apparatus 100 forms an optical image of a subject on the pixel array of the imaging unit 112 using the photographing lens 111, but the photographing lens 111 may be detachable from the body 123 of the image processing apparatus 100. However, it may be detachable. The image processing apparatus 100 writes and reads image data to and from the recording medium 120 via the recording medium control unit 119. However, the recording medium 120 may be detachable from the image processing apparatus 100. However, it may be non-detachable.

  The CPU 101 controls the operation of each unit (each functional block) of the image processing apparatus 100 via the internal bus 130 by executing a computer program stored in the nonvolatile memory 103. As will be described later, the CPU 101 can function as a setting unit that sets a selection area 122 for selecting at least a part of an image acquired by the imaging unit 112. In addition, the CPU 101, that is, the setting unit sets the size of the selection area 122 so that it fits on the display screen 125. In addition, the CPU 101, that is, the setting unit, keeps the size of the selection region 122 constant even when the posture of the imaging unit 112 changes under a predetermined condition. In addition, the CPU 101, that is, the setting unit changes the selection area 122 when a group of pictures including a plurality of images is switched. In addition, the CPU 101, that is, the setting unit, can perform the operation in the vertical imaging mode, specifically, the setting of the selection area 122, etc., when the display screen 125 faces the subject. Further, the CPU 101, that is, the setting unit, can perform an operation in the vertical shooting mode when communication with the external device 127 is established. The CPU 101 can also function as a file generation unit. Such file generation means can generate the first image group by sequentially encoding the tiles 126 corresponding to the selected area 122. In addition, the file generation unit can generate the second image group by sequentially encoding the tiles 126 corresponding to at least the area other than the selected area 122.

  The memory 102 is a rewritable volatile memory. The memory 102 temporarily records a computer program for controlling the operation of each unit of the image processing apparatus 100, information such as parameters relating to the operation of each unit of the image processing apparatus 100, information received by the communication control unit 117, and the like. . The memory 102 temporarily records an image acquired by the imaging unit 112, an image processed by the image processing unit 113, the encoding processing unit 114, and the like, information, and the like. The memory 102 has a sufficient storage capacity for temporarily recording these.

  The nonvolatile memory 103 is an electrically erasable and recordable memory. For example, an EEPROM or the like is used. The nonvolatile memory 103 stores information such as a computer program that controls the operation of each unit of the image processing apparatus 100 and parameters related to the operation of each unit of the image processing apparatus 100. By such a computer program, various operations performed by the image processing apparatus 100 according to the present embodiment are realized.

  The operation unit 104 provides a user interface for operating the image processing apparatus 100. The operation unit 104 includes various buttons such as a power button, a menu button, and a shutter button, and the various buttons include switches, a touch panel, and the like. The CPU 101 controls the image processing apparatus 100 in accordance with a user instruction input via the operation unit 104. Here, the case where the CPU 101 controls the image processing apparatus 100 based on an operation input via the operation unit 104 has been described as an example, but the present invention is not limited to this. For example, the CPU 101 may control the image processing apparatus 100 based on a request input via a communication unit 118 from a remote controller (not shown), a portable terminal (not shown), or the like.

  The photographing lens (lens unit) 111 includes a lens group (not shown) including a zoom lens and a focus lens, a lens control unit (not shown), a diaphragm (not shown), and the like. The photographic lens 111 can function as zoom means for changing the angle of view, as will be described later. The lens control unit performs focus adjustment and aperture value (F value) control by a control signal transmitted from the CPU 101. The imaging unit 112 can function as an acquisition unit that sequentially acquires a plurality of images constituting a moving image. As the imaging unit 112, for example, an area image sensor such as a CCD (charge coupled device) or a CMOS (complementary metal oxide semiconductor) device is used. The imaging unit 112 has a pixel array (not shown) in which photoelectric conversion units (not shown) that convert an optical image of a subject into an electric signal are arranged in a matrix, that is, two-dimensionally. An optical image of the subject is formed on the pixel array by the photographing lens 111. The imaging unit 112 outputs the captured image to the image processing unit 113 or the memory 102. Note that the imaging unit 112 can also acquire a still image.

  The image processing unit 113 performs predetermined image processing on the image data output from the imaging unit 112 or the image data read from the memory 102. Examples of such image processing include interpolation processing, reduction processing (resizing processing), color conversion processing, and the like. In addition, the image processing unit 113 performs predetermined calculation processing for exposure control, distance measurement control, and the like, using the image data acquired by the imaging unit 112. Based on the calculation result obtained by the calculation processing by the image processing unit 113, the CPU 101 performs exposure control, distance measurement control, and the like. Specifically, the CPU 101 performs AE (automatic exposure) processing, AWB (auto white balance) processing, AF (autofocus) processing, and the like.

  The encoding processing unit 114 compresses the size of the image data by performing intra-frame prediction encoding (intra-screen prediction encoding), inter-frame prediction encoding (inter-frame prediction encoding), and the like on the image data. . The encoding processing unit 114 is, for example, H.264. The encoding process is performed according to the H.265 (ITU H.265 or ISO / IEC 23008-2) method. As will be described later, the encoding processing unit 114 can function as a dividing unit that divides an image into a plurality of tiles 126 based on the setting of the selection region 122. In addition, the encoding processing unit 114 can function as an encoding unit that encodes the tile 126 corresponding to the selection region 122, as will be described later. In addition, the encoding processing unit 114 can also encode the tile 126 corresponding to an area other than the selected area 122, as will be described later. In addition, as described later, the encoding processing unit 114 can use predetermined data for the tile 126 corresponding to a portion other than the selection region 122 when the posture of the imaging unit 112 changes. Further, as will be described later, the encoding processing unit 114 scans the selection region 122 in the same direction without depending on the posture of the imaging unit 112 when the image is captured. However, rotation of image data and the like are appropriately performed in advance by the CPU 101. The details of the encoding processing unit 114 will be described later with reference to FIG.

  The display control unit 115 controls the display unit 116. The display unit 116 includes a display screen 125. The display control unit 115 generates an image that can be displayed on the display screen 125 of the display unit 116 by performing resizing processing, color conversion processing, and the like on the image data, and displays the image, that is, the image signal. Output to. The display unit 116 displays an image on the display screen 125 based on the image signal sent from the display control unit 115. The display unit 116 has an OSD (On Screen Display) function that is a function for displaying a setting screen such as a menu on the display screen 125. The display control unit 115 can superimpose the OSD image on the image signal and output it to the display unit 116. The display unit 116 is, for example, a vari-angle display unit, and can rotate, for example, 180 degrees. Information indicating that the display unit 116 is rotated, for example, 180 degrees is notified from the display unit 116 to the CPU 101 via the display control unit 115. Note that when the display unit 116 is rotated, for example, 180 degrees, the image processing apparatus 100 can be used, for example, for self-portrait, that is, for self-portrait. Note that self-shooting, that is, self-shooting, means that a photographer takes a picture of himself as a subject. The display unit 116 is configured by, for example, a liquid crystal display or an organic EL. The display unit 116 may be a touch panel, for example. When the display unit 116 is a touch panel, the display unit 116 can also function as the operation unit 104. The display control unit 115 can function as display control means for displaying the selection area 122 on the display screen 125. As described above, the selection area 122 is set to a vertically long rectangle, for example. As will be described later, when the display screen 125 is in the horizontally long state, the display unit 116 displays the vertically long selection area 122 on the horizontally long display screen 125. Further, as will be described later, when the display screen 125 is in the vertically long state, the display unit 116 displays the vertically long selection area 122 on the display screen 125 in the vertically long state.

  The communication control unit 117 is controlled by the CPU 101. The communication control unit 117 generates a modulation signal that conforms to a wireless communication standard predetermined by IEEE 802.11 or the like, for example, and outputs the modulation signal to the communication unit 118. In addition, the communication control unit 117 receives a modulated signal conforming to the wireless communication standard via the communication unit 118, decodes the received modulated signal, and outputs a signal corresponding to the decoded signal to the CPU 101. The communication control unit 117 includes a register for storing communication settings. The communication control unit 117 can adjust the transmission / reception sensitivity during communication under the control of the CPU 101. The communication control unit 117 can perform transmission / reception with a predetermined modulation method. The communication unit 118 includes an antenna that outputs the modulation signal supplied from the communication control unit 117 to the outside and receives the modulation signal from the outside. The communication unit 118 includes a communication circuit and the like. Here, the case where wireless communication is performed by the communication unit 118 has been described as an example, but the communication performed by the communication unit 118 is not limited to wireless communication. For example, the communication unit 118 and the external device 127 may be connected by electrical connection using wiring or the like.

  The recording medium control unit 119 controls the recording medium 120. The recording medium control unit 119 outputs a control signal for controlling the recording medium 120 to the recording medium 120 based on a request from the CPU 101. As the recording medium 120, for example, a nonvolatile memory or a magnetic disk is used. As described above, the recording medium 120 may be detachable or non-detachable. The recording medium 120 records encoded image data and the like. Image data or the like is saved as a file in a format compatible with the file system of the recording medium 120. Examples of such files include MP4 files (ISO / IEC 14496-14: 2003) and MXF (Material eXchange Format) files.

  The tilt detection unit (sensor tilt detection unit) 121 detects the tilt (posture) of the imaging unit 112. As the inclination detection unit 121, for example, an acceleration sensor is used. As the acceleration sensor, various types of acceleration sensors such as a capacitance detection method and a piezoresistive method can be used as appropriate. The capacitance detection type acceleration sensor detects a change in capacitance between the sensor element movable portion and the fixed portion. A piezoresistive acceleration sensor detects strain generated in a spring portion due to acceleration by a piezoresistive element disposed in a spring portion that connects a sensor element movable portion and a fixed portion. The respective functional blocks 101 to 104, 112 to 115, 117, 119 and 121 can be accessed with each other via the internal bus 130.

<Encoding processing unit>
FIG. 3 is a block diagram illustrating the encoding processing unit 114 provided in the image processing apparatus 100 according to the present embodiment. In FIG. 3, the encoding processing unit 114 and the memory 102 are extracted and shown. The encoding processing unit 114 is, for example, H.264, which is one of moving image compression standards. Encoding is performed according to the H.265 encoding method.

  The image dividing unit 301 divides an image to be encoded into a plurality of rectangular regions, that is, a plurality of tiles based on the selection region 122. H. In the H.265 encoding method, it is possible to perform screen division using tiles. If tiles are used, an image can be divided vertically and horizontally at an arbitrary position. In the tile, reference to pixel values such as intra-frame prediction and inter-frame prediction is interrupted. That is, the tile is a rectangular area that can be decoded without referring to the pixel values of other tiles, that is, other rectangular areas. For this reason, each tile can be used independently. Information about the position of the selection region 122 is given from the CPU 101 to the image dividing unit 301. The image dividing unit 301 divides an image to be encoded into a plurality of tiles based on information about the selection area 122 given from the CPU 101. In the screen division, a horizontal tile division number num_tile_columns_minus1, a vertical tile division number num_tile_rows_minus1, and the like are set as appropriate. A tile structure fixed flag tiles_fixed_structure_flag indicating whether the tile structure is fixed or arbitrary is set to 0 indicating that the tile structure is variable, for example. The size of the rectangular area, that is, the tile size is represented by the number of pixels in the horizontal direction and the number of pixels in the vertical direction. However, the size of the rectangular area, that is, the size of the tile can be specified by the number of pixels corresponding to one side of the rectangular area and the aspect ratio. The image dividing unit 301 assigns a number to each tile defined by the screen division, that is, a tile number. The position of each tile in the image to be encoded can be specified by the tile number. Note that the size of each tile can also be determined by information indicating the number of CTBs (Cording Tree Blocks), for example. The image division unit 301 outputs the prediction encoding condition for each tile to the prediction encoding method determination unit 302 during the encoding process. Such prediction encoding conditions include information that defines a range in which a prediction pixel block of the target pixel block (encoding target block) is acquired when encoding the target pixel block in a certain tile. This point will be described in detail later.

  The prediction encoding method determination unit 302 determines a prediction encoding method for each pixel block in the encoding target region based on the screen division information input from the image division unit 301 and the prediction encoding condition. The predictive encoding method determination unit 302 performs simple inter-frame prediction or inter-frame prediction processing including motion detection from the input image signal and the encoded pixel value read from the memory 102, An evaluation value indicating the coding efficiency is calculated. Then, the predictive encoding method determination unit 302 determines the predictive encoding scheme that provides the best encoding efficiency. When the pixel block to be encoded is an I slice, the prediction encoding method determination unit 302 determines the block size and prediction mode of the prediction pixel in the frame. When the encoding pixel block is a P slice or a B slice, the one with higher encoding efficiency is selected from among intra-frame prediction and inter-frame prediction. A frame in which all slices in the frame are I slices is called an I frame, a frame in which all slices in the frame are P slices is called a P frame, and all slices in the frame are B slices. A certain frame is called a B frame. In the case of intra-frame prediction, the predictive encoding method determination unit 302 determines intra-frame predictive encoding parameters such as a block size of a predicted pixel in a frame and an intra-frame prediction mode. Also, in the case of inter-frame prediction, the predictive coding method determining unit 302 determines inter-frame predictive coding parameters such as a reference frame, a pixel block division pattern, and a motion vector. The predictive coding method determining unit 302 outputs the determined predictive coding parameter to the predictive coding processing unit 303.

  The predictive coding processing unit 303 operates as follows when coding the pixel block of interest in the tile of interest in the encoding target frame. That is, in this case, the predictive encoding processing unit 303 generates a predictive pixel block from the encoded image read from the memory 102 in accordance with the predictive encoding parameter determined by the predictive encoding method determining unit 302. To do. Then, the predictive coding processing unit 303 outputs a prediction residual block that is a difference between the target pixel block and the predicted pixel block to the orthogonal transform / quantization unit 305. Further, the predictive coding processing unit 303 also outputs the predicted pixel block to the local decoding unit 306.

  The orthogonal transform / quantization unit 305 performs orthogonal transform processing on the prediction residual block supplied from the predictive coding processing unit 303. Further, the orthogonal transform / quantization unit 305 quantizes the coefficient obtained by the orthogonal transform process using a quantization step according to the quantization parameter set by the code amount control unit 307. The orthogonal transform / quantization unit 305 outputs the quantized coefficient, that is, the quantized data, to the entropy encoding unit 308 and the local decoding unit 306.

  The local decoding unit 306 generates prediction residual data by performing inverse quantization processing and inverse orthogonal transformation processing on the quantized data input from the orthogonal transformation / quantization unit 305. Then, the local decoding unit 306 performs decoding processing by adding the prediction image input from the prediction encoding processing unit 303 to the generated prediction residual data, and indicates the pixel block obtained by the decoding processing. The image data to be stored is stored in the memory 102. The decoded image data stored in the memory 102 is used for intra-frame prediction processing. Further, the decoded data that has been subjected to the deblocking filter processing is held in the memory 102. The decoded data after the deblocking filter process held in the memory 102 is also used for the inter-frame prediction process.

  The code amount control unit 307 controls the code amount of the encoded data so that the encoded picture buffer does not overflow or underflow. The code amount control unit 307 generates a quantization parameter for the subsequent frame based on the generated code amount after entropy encoding supplied from the entropy encoding unit 308, and orthogonally transforms / quantizes the generated quantization parameter. Supplied to the unit 305.

  The entropy coding unit 308 performs entropy coding processing by CABAC (context adaptive binary arithmetic coding) on the input quantized data in units of slices. The entropy encoding unit 308 includes a binarizing unit that converts input multi-level quantized data into binary data, and a binarized data memory that stores binarized data generated by the binarizing unit And have. The entropy encoding unit 308 includes a context calculation unit that calculates the occurrence probability of the binarized data according to the context and holds the occurrence probability of the binary data obtained by the calculation. The entropy encoding unit 308 includes an arithmetic encoding unit that performs arithmetic encoding according to the occurrence probability of binary data supplied from the context calculation unit. The entropy encoding unit 308 outputs the encoded data to the multiplexing processing unit 309 in units of tiles, and outputs the generated code amount after entropy encoding to the code amount control unit 307.

  At the start of image encoding, the multiplexing processing unit 309 sends information such as the size of the frame to be encoded, the tile division position, the tile size, the tile number, and the tile number where the main subject is located to the file header. To store. Note that if it is common to both the encoding and decoding apparatuses that tile numbers are assigned in the order of raster scanning in units of tiles, the tile numbers may be unnecessary.

<Display example of selection area>
FIG. 4 is a diagram illustrating a display example of the selection area. FIG. 4 shows a state in which the selection area 122 is displayed on the preview screen. FIG. 4A to FIG. 4C illustrate an example in which shooting is performed in a state where the imaging unit 112 is horizontally long, that is, a horizontally long image is acquired by the imaging unit 112. In the example illustrated in FIG. 4A, a horizontally long image acquired by the imaging unit 112, that is, the entire image 124 is displayed on the display screen 125 of the display unit 116, and a vertically long rectangle is included in the range of the entire image 124. A selection area 122 is displayed. In the example shown in FIG. 4A, the size of the long side of the selection region 122 and the size of the short side of the entire image 124 are the same. Thus, the size of the selection area 122 is set so as to fit on the display screen 125. In the example shown in FIG. 4A, the selection area 122 can be moved left and right in accordance with an instruction from the user. Further, in the example shown in FIG. 4A, the selection region 122 can be expanded in the horizontal direction. In addition, in the example shown in FIG. 4A, the selection area 122 can be reduced. Note that movement, enlargement, reduction, and the like of the selection area 122 may be performed by a user operating an operation button or the like included in the operation unit 104, or may be performed by a user operating a touch panel. The reduction direction of the selection region 122 may be only the vertical direction, may be only the horizontal direction, or may be both the vertical and horizontal directions. The entire image 124 and the selection region 122 may be similar or may not be similar.

  FIG. 4B shows an example in which the size of the selection area 122 is reduced. In the example shown in FIG. 4B, the size of the long side of the selection region 122 is smaller than the size of the short side of the entire image 124. For this reason, in the example shown in FIG. 4B, the selection area 122 can be moved both in the vertical direction and in the horizontal direction in accordance with an instruction from the user. Further, in the example shown in FIG. 4B, the selection area 122 can be enlarged or reduced. Note that the selection area 122 may be enlarged or reduced only in the vertical direction, only in the horizontal direction, or in both the vertical and horizontal directions. Further, as described above, the whole image 124 and the selection region 122 may be similar or may not be similar.

  FIG. 4C shows a case where the selection area 122 can be selected from a plurality of candidates. Here, a case where three selection regions 122, that is, selection regions 122a to 122c are displayed on the display screen 125 of the display unit 116 is shown as an example. It should be noted that reference numerals 122a to 122c are used when describing individual selection areas, and reference numeral 122 is used when general selection areas are described. One of the plurality of selection areas 122a to 122c is selected by an instruction from the user. The plurality of selection regions 122a to 122c may be similar or may not be similar. Further, as described above, the whole image 124 and the selection region 122 may be similar or may not be similar. In addition, the selection area 122 selected from the plurality of selection areas 122a to 122c may be moved, or may be enlarged or reduced.

  FIG. 4D illustrates an example in which shooting is performed in a state where the imaging unit 112 is in a vertically long posture, that is, a vertically long image is acquired by the imaging unit 112. In the example shown in FIG. 4D, the size of the long side of the selection area 122 and the size of the short side of the entire image 124 are the same. In the example shown in FIG. 4D, as in the example shown in FIG. 4B, the selection area 122 can be moved both in the vertical direction and in the horizontal direction by an instruction from the user. In addition, the selection area 122 can be enlarged or reduced. Note that the selection area 122 may be enlarged or reduced only in the vertical direction, only in the horizontal direction, or in both the vertical and horizontal directions. Further, as described above, the whole image 124 and the selection region 122 may be similar or may not be similar. In addition, since the entire image 124 is vertically long like the selection region 122, the size of the selection region 122 and the size of the entire image 124 match if the selection region 122 is maximized. In the example shown in FIG. 4D, a portion of the entire image 124 other than the selected region 122 is displayed on the display screen 125, but a portion of the entire image 124 other than the selected region 122 is photographed. You may make it not display on the display screen 125 as a target. In addition, an image using predetermined data, that is, a predetermined image may be displayed in a portion other than the selection area 122 in the entire image 124. Examples of such predetermined data include a black image. That is, a portion other than the selection region 122 in the entire image 124 may be filled with, for example, black. Further, when the posture of the imaging unit 112 is changed, predetermined data may be used for a portion other than the selection region 122. For example, when the imaging unit 112 changes from a horizontally long state to a vertically long state, predetermined data may be used for a portion other than the selection region 122. Further, the selection region 122 may be selected from a plurality of candidates. Furthermore, the selection area 122 selected from among a plurality of candidates may be moved, or may be enlarged or reduced.

<Tile composition>
FIG. 5 is a diagram illustrating an example of a tile configuration. FIG. 5A illustrates a case where an image is acquired with the imaging unit 112 in a horizontally long state, that is, an image is acquired with the long side of the pixel array included in the imaging unit 112 being horizontal. An example is shown. In the example of FIG. 5A, the image acquired by the imaging unit 112, that is, the entire image 124 is divided into three tiles 126. Note that reference numerals 126A1, 126A2, and 126B are used when describing individual tiles, and reference numerals 126 are used when general tiles are described. The tile 126B corresponds to the selection area 122. The tile 126A1 and the tile 126A2 correspond to portions other than the selection area 122 in the entire image 124. The direction of the arrow in FIG. 5A indicates the scan direction of the raster scan performed at the time of encoding. For example, the raster scan is performed in the order of the tile 126A1, the tile 126A2, and the tile 126B. The encoding process is performed in units of CTU (Coding Tree Unit).

  FIG. 5B shows a case where an image is acquired in a state where the imaging unit 112 is vertically long, that is, a case where an image is acquired in a state where the long side of the pixel array provided in the imaging unit 112 is vertical. Show. In the example of FIG. 5B, the image acquired by the imaging unit 112, that is, the entire image 124 is divided into five tiles. Note that reference numerals 126C1, 126C2, 126C3, 126C4, and 126D are used when describing individual tiles, and reference numerals 126 are used when general tiles are described. The tile 126D corresponds to the selection area 122. The tile 126C1, the tile 126C2, the tile 126C3, and the tile 126C4 correspond to portions other than the selection region 122 in the entire image 124. The direction of the arrow in FIG. 5B indicates the scan direction of the raster scan performed at the time of encoding. For example, the raster scan is performed in the order of the tile 126C1, the tile 126C2, the tile 126C3, the tile 126C4, and the tile 126D. As can be seen from FIGS. 5A and 5B, the selection region 122 is scanned in the same direction without depending on the posture of the imaging unit 112 when the image is captured. However, rotation or the like of image data is appropriately performed before encoding.

  As described above, the mode of image division acquired in the state where the imaging unit 112 is horizontally long is different from the mode of image division acquired in the state where the imaging unit 112 is vertically long. Note that the size of the tile 126B shown in FIG. 5A and the size of the tile 126D shown in FIG. 5B may or may not be the same. When shooting is started with the image pickup unit 112 set in the landscape orientation, and then the image pickup unit 112 is set in the portrait orientation, shooting can be continued without changing the size of the tile 126D and the size of the tile 126B. Is possible. In such a case, even when the posture of the imaging unit 112 is changed, the size of the selection region 122 is kept constant. The configuration of the tile 126 is not limited to the example illustrated in FIGS. 5A and 5B. For example, the selection area 122 may be configured by a plurality of tiles 126.

<Operation of Image Processing Device>
FIG. 6 is a flowchart showing the operation of the image processing apparatus 100 according to the present embodiment. The program stored in the non-volatile memory 103 is expanded in the memory 102 and the program is executed by the CPU 101, whereby the operation of the image processing apparatus 100 according to the present embodiment is realized.

  In step S601, the CPU 101 determines whether or not the shooting mode is the vertical shooting mode. As described above, the vertical shooting mode is an operation mode in which a vertically long image is acquired regardless of whether the imaging unit 112 is in a horizontally long state or the imaging unit 112 is in a vertically long state. Whether or not the vertical shooting mode is set is determined based on the mode setting value recorded in the nonvolatile memory 103. If it is the vertical shooting mode (YES in step S601), the process proceeds to step S602. If the vertical shooting mode is not set (NO in step S601), the CPU 101 records information indicating that the selection area 122 is not set in the memory 102, and the process proceeds to step S603. In the following cases, the operation in the vertical shooting mode may be performed. That is, the operation in the vertical shooting mode may be performed when communication with the external device 127 is established. In this case, examples of the external device 127 include a device for self-shooting, more specifically, a self-shooting stick. A self-shooting stick is a stick-shaped device that allows a photographer to take a picture of himself as a subject. The image processing apparatus 100 according to the present embodiment is provided at the tip of the self-taking stick. For example, when communication between the selfie stick and the image processing apparatus 100 is established via the communication control unit 117 and the communication unit 118, the operation in the vertical shooting mode may be executed. Further, when the display screen 125 of the display unit 116 faces the subject, that is, the photographer, the operation in the vertical shooting mode may be performed.

  In step S <b> 602, the CPU 101 controls the display control unit 115 to display the selection area 122 as described above with reference to FIG. 4 on the display screen 125 of the display unit 116. When the selection area 122 is selected from a plurality of candidates, the plurality of candidates for the selection area 122 are displayed on the display screen 125 of the display unit 116. In this case, the CPU 101 selects one of the selection regions 122 based on a selection instruction given by the user via the operation unit 104. Further, the CPU 101 changes the selection area 122 based on an instruction to change the selection area 122 performed by the user via the operation unit 104. Such changes include movement, enlargement, reduction, and the like. The CPU 101 changes the selection area 122 when a group of pictures including a plurality of images is switched. Thus, the CPU 101 sets the selection area 122 based on an instruction from the user via the operation unit 104. The CPU 101 records information about the position of the selection area 122 in the memory 102. Thereafter, the process proceeds to step S603.

  In step S603, the CPU 101 determines whether to perform shooting. Whether or not shooting is to be executed is determined based on whether or not an image recording start button included in the operation unit 104 has been pressed. If the image recording start button has been pressed (YES in step S603), the process proceeds to step S604. If the image recording start button has not been pressed (NO in step S603), step S603 is repeated.

  In step S <b> 604, the CPU 101 performs setting at the previous stage of the encoding process performed by the encoding processing unit 114, that is, setting at the preparation stage of the encoding process. Specifically, for example, the CPU 101 sets the configuration of the tile 126 as described above with reference to FIG. The CPU 101 also determines what file is to be recorded. In the first mode, only the file including the image group for the selection area 122 is recorded. In this case, one file is generated. The second mode is to record a first file including an image group for the selected area 122 and a second file including an image group for an area other than the selected area 122. In this case, Two files are generated. In the third mode, a first file including an image group for the selected area 122 and a second file including an image group for the entire image 124 are recorded. In this case, two files are recorded. Generated. Further, the CPU 101 sets various registers for performing the encoding process, etc., so that the encoding processing unit 114 can be operated. Thereafter, the process proceeds to step S605.

  In step S605, the CPU 101 notifies the encoding processing unit 114 of the address of the image data stored in the memory 102 by the image processing unit 113, and instructs the encoding processing unit 114 to execute the encoding process. The details of the encoding process performed in the encoding processing unit 114 will be described later with reference to FIG. Thereafter, the process proceeds to step S606.

  In step S606, the CPU 101 records the encoded image and syntax information obtained by the encoding processing unit 114 on the recording medium 120 as stream data via the recording medium control unit 119. In the case of the first aspect described above, that is, when only a file including an image group for the selection area 122 is recorded, an encoded image corresponding to the selection area 122, syntax information regarding the encoded image, and Are recorded on the recording medium 120 as stream data. In the case of the second mode described above, that is, when the first file including the image group for the selected area 122 and the second file including the image group for the area other than the selected area 122 are recorded, The following operations are performed. In this case, the encoded image corresponding to the selected area 122, the syntax information regarding the encoded image, the encoded image corresponding to the area other than the selected area 122, and the syntax information for the encoded image. Are recorded on the recording medium 120 as stream data. In the case of the third mode described above, that is, when the first file including the image group for the selected area 122 and the second file including the image group for the entire image 124 are recorded, the following is performed. Operation is performed. In this case, the encoded image corresponding to the selected area 122, the syntax information regarding the encoded image, the encoded image corresponding to the entire image 124, and the syntax information regarding the encoded image are as follows: It is recorded on the recording medium 120 as stream data. Thereafter, the process proceeds to step S607.

  In step S <b> 607, the CPU 101 determines whether shooting has ended. The CPU 101 determines whether or not shooting has been completed based on a signal from the operation unit 104 operated by the user. If CPU 101 determines that shooting has ended (YES in step S607), the process proceeds to step S608. If the CPU 101 determines that shooting has not ended (NO in step S607), the process proceeds to step S601.

  In step S <b> 608, the CPU 101 performs an encoding process end process on the encoding processing unit 114. Specifically, a value indicating the end of the encoding process is recorded in a register (not shown) provided in the encoding processing unit 114. Thereafter, the process proceeds to step S609.

  In step S <b> 609, the CPU 101 generates a file including the stream data recorded on the recording medium 120, that is, the image group encoded by the encoding processing unit 114, via the recording medium control unit 119. As a result, the moving image data recorded on the recording medium 120 can be read as a file. In the case of the first mode described above, for example, one file is generated, and in the case of the second mode and the third mode described above, for example, two files are generated. Thus, the operation of the image processing apparatus 100 according to the present embodiment is completed. Note that the CPU 101 may transmit the file thus obtained, that is, the moving image file, to the external device 127 via the communication control unit 117 and the communication unit 118. In this case, examples of the external device 127 include a device that can record a file. When the CPU 101 transmits the file to the external device 127, the file may be recorded on the recording medium 120 or may not be recorded.

<Encoding process>
FIG. 7 is a flowchart illustrating an example of the encoding process performed by the image processing apparatus 100 according to the present embodiment. The encoding process shown in FIG. 7 corresponds to step S605 in FIG. 6, and is executed by an instruction from the CPU 101. Here, in the case of the third aspect described above, that is, when the first file including the image group for the selected area 122 and the second file including the image group for the entire image 124 are recorded. Will be described as an example.

  In step S <b> 701, the encoding processing unit 114 performs image division processing using the image dividing unit 301. Specifically, the encoding processing unit 114 sets the selection area 122 as the tile 126 based on the position information of the selection area 122 recorded in the memory 102 in step S602. For example, in the case of FIG. 5A, the selection area 122 is set as the tile 126B. In the case shown in FIG. 5B, the selection area 122 is set as the tile 126D. In addition, the encoding processing unit 114 sets a portion other than the selection region 122 as the tile 126. For example, in the case of FIG. 5A, areas other than the selection area 122 are set as tiles 126A1 and 126A2. In the case shown in FIG. 5B, areas other than the selected area 122 are set as tiles 126C1, 126C2, 126C3, and 126C4. The encoding processing unit 114 records the area information for each tile 126 set in this way in the memory 102. Thereafter, the process proceeds to step S702.

  In step S702, the encoding processing unit 114 sets a value in the quantization table set in the code amount control unit 307, that is, a quantization value. The quantized values for the tiles 126B and 126D corresponding to the selection area 122 may be set small so that sufficient image quality can be obtained. On the other hand, the quantized values for the tiles 126A1, 126A2, 126C1, 126C2, 126C3, and 126C4 corresponding to the areas other than the selected area 122 may be set large to suppress the code amount. Thereafter, the process proceeds to step S703. The setting of the quantization value is not limited to such an aspect. For example, the quantization value may be set small for a region where it is desired to obtain a sufficient code amount.

  In step S703, the encoding processing unit 114 executes an encoding process on the image. Details of the encoding process have been described above with reference to FIG. The encoding processing unit 114 stores the encoded image data in the memory 102. Thereafter, the process proceeds to step S704.

  In step S704, the encoding processing unit 114 includes various information, that is, various information, in the syntax information. The syntax information used for the first file including the image group for the selection area 122 includes information indicating the configuration of the tile 126, range information of the tile 126 corresponding to the selection area 122, and the like. The syntax information used for the second file including the image group for the entire image 124 includes information indicating the configuration of the tile 126, information indicating the presence / absence of an image corresponding to the selection area 122, and a tile corresponding to the selection area 122. 126 range information and the like are included. Then, the encoding processing unit 114 stores the syntax information in the memory 102. Here, the case where the information indicating the presence / absence of the image corresponding to the selection area 122 and the range information of the tile 126 corresponding to the selection area 122 is included in the syntax information has been described as an example, but the present invention is not limited to this. Absent. For example, these pieces of information may be included in the container information. Thus, the encoding process is performed by the image processing apparatus 100 according to the present embodiment.

  Note that the encoding process as shown in FIG. 7 may be performed by the CPU 101. Further, here, the case where the encoded data corresponding to the selection region 122 and the encoded data for the entire image 124 are generated has been described as an example, but the present invention is not limited to this. For example, only the encoded data for the entire image 124 may be generated without generating the encoded data for the selected region 122. Alternatively, only the encoded data for the selected region 122 may be generated without generating the encoded data for the entire image 124. Further, the tile 126 may include a slice.

  Thus, according to the present embodiment, the image is divided based on the setting of the selection area, and the tile corresponding to the selection area is encoded. For this reason, an image corresponding to the selected region can be obtained. If a tile corresponding to an area other than the selected area is also encoded, not only the image corresponding to the selected area but also the entire image can be obtained. Therefore, according to the present embodiment, encoded data of both a horizontal image and a vertical image can be obtained. Therefore, according to the present embodiment, it is possible to provide an image processing apparatus that can satisfactorily acquire a predetermined image and an image of a selected area.

[Second Embodiment]
An image processing apparatus and an image processing method according to the second embodiment will be described in detail with reference to the drawings. The same components as those in the image processing apparatus and the image processing method according to the first embodiment shown in FIGS. 1 to 7 are denoted by the same reference numerals, and description thereof is omitted or simplified.
The image processing apparatus according to the present embodiment performs processing based on the inclination (posture) of the imaging unit 112. FIG. 8 is a flowchart showing the operation of the image processing apparatus 100 according to the present embodiment. The program stored in the non-volatile memory 103 is expanded in the memory 102, and the program is executed by the CPU 101, whereby the operation of the image processing apparatus 100 according to the present embodiment is realized.

  In step S801, the CPU 101 determines whether or not the shooting mode is the vertical shooting mode, as in step S601 described above with reference to FIG. If it is the vertical shooting mode (YES in step S801), the process proceeds to step S802. If the vertical shooting mode is not set (NO in step S801), the CPU 101 records information indicating that the selection area 122 is not set in the memory 102, and the process proceeds to step S804.

  In step S <b> 802, the CPU 101 acquires information indicating the tilt of the imaging unit 112 by the tilt detection unit 121, and stores information indicating the tilt of the imaging unit 112, i.e., sensor tilt information, in the memory 102. Thereafter, the process proceeds to step S803.

  In step S803, the CPU 101 displays the selection region 122 on the display screen 125 of the display unit 116 based on the sensor tilt information stored in the memory 102 in step S802. As described above, the selection area 122 is displayed by the CPU 101 controlling the display control unit 115. Specifically, when the sensor tilt information indicates that the imaging unit 112 is in the horizontally long state, the CPU 101 selects the selection area as shown in, for example, any one of FIGS. 4A to 4C. 122 is displayed. When the sensor tilt information indicates that the imaging unit 112 is in the vertically long state, the CPU 101 displays the selection area 122 as shown in FIG. 4D, for example. In the example shown in FIG. 4D, a portion of the entire image 124 other than the selected region 122 is displayed on the display screen 125, but a portion of the entire image 124 other than the selected region 122 is photographed. You may make it not display on the display screen 125 as a target. Further, a specific image may be displayed in a portion other than the selection area 122 in the entire image 124. An example of such a specific image is a black image. That is, a portion other than the selection region 122 in the entire image 124 may be filled with, for example, black. Further, the selection region 122 may be selected from a plurality of candidates. Furthermore, the selected selection area 122 may be moved, or may be enlarged or reduced. In addition, the CPU 101 may not display the selection area 122 when the imaging unit 112 is in the vertically long state. The CPU 101 records the position information of the selection area 122 in the memory 102 based on the setting of the selection area 122 performed by the user via the operation unit 104. Thereafter, the process proceeds to step S804.

  In step S <b> 804, the CPU 101 determines whether or not to perform shooting, similar to step S <b> 603 described above with reference to FIG. 6. If the image recording start button has been pressed (YES in step S804), the process proceeds to step S805. If the image recording start button has not been pressed (NO in step S804), step S804 is repeated.

  In step S805, the CPU 101 determines whether or not the imaging unit 112 is in a vertically long state based on the sensor tilt information stored in the memory 102 in step S802. If the imaging unit 112 is in the vertically long state (YES in step S805), the CPU 101 proceeds to step S806. If the imaging unit 112 is not in the vertically long state (NO in step S805), the process proceeds to step S807.

  In step S <b> 806, the CPU 101 enlarges or reduces the selection area 122 according to the change in the angle of view of the photographing lens 111. Note that the angle of view is changed by controlling a zoom lens provided in the photographing lens 111. For example, when the zoom lens of the photographic lens 111 has reached the end on the telephoto side, the CPU 101 enlarges the size of the selection area 122 displayed on the display unit 116. Since the imaging unit 112 is in a vertically long state, the size of the selection area 122 can be expanded to the size of the entire image 124. Thereafter, the process proceeds to step S807.

  In step S <b> 807, the CPU 101 acquires information indicating the tilt of the imaging unit 112 by the tilt detection unit 121, and stores information indicating the tilt of the imaging unit 112, that is, sensor tilt information in the memory 102. Then, the CPU 101 determines whether or not the sensor tilt information stored in the previous memory 102 matches the sensor tilt information stored in the current memory 102. If they do not match, the inclination of the imaging unit 112 is changed during shooting. If the sensor tilt information stored in the previous memory 102 does not match the sensor tilt information stored in the current memory 102 (YES in step S807), the CPU 101 proceeds to step S808. If the sensor tilt information stored in the previous memory 102 matches the sensor tilt information stored in the current memory 102 (NO in step S807), the CPU 101 proceeds to step S809.

  In step S808, the CPU 101 performs the previous setting for performing the encoding process by the encoding processing unit 114, similarly to step S604 described above with reference to FIG. At this time, the CPU 101 determines whether the imaging unit 112 has changed from the horizontally long state to the vertically long state based on the sensor tilt information. When a change from the horizontally long state to the vertically long state occurs in the imaging unit 112, the CPU 101 encodes information indicating that the change from the horizontally long state to the vertically long state occurs in the image pickup unit 112. Set in the register of the processing unit 114. The CPU 101 sets the configuration of the tile 126 as described above with reference to FIG. 5 according to the attitude of the imaging unit 112. Thereafter, the process proceeds to step S809.

  In step S809, the CPU 101 executes an encoding process in the same manner as in step S605 described above with reference to FIG. Details of the encoding process performed in the encoding processing unit 114 will be described later with reference to FIG. Thereafter, the process proceeds to step S810.

  In step S810, as in step S606 described above with reference to FIG. 6, the CPU 101 converts the encoded image generated by the encoding processing unit 114 and syntax information into stream data via the recording medium control unit 119. Recording is performed on the recording medium 120. Thereafter, the process proceeds to step S811.

  In step S811, the CPU 101 determines whether or not shooting has been completed, as in step S607 described above with reference to FIG. If CPU 101 determines that shooting has ended (YES in step S811), the process proceeds to step S812. If the CPU 101 determines that shooting has not ended (NO in step S811), the process proceeds to step S801.

  In step S812, the CPU 101 performs an encoding process end process on the encoding processing unit 114 in the same manner as in step S608 described above with reference to FIG. Thereafter, the process proceeds to step S813.

  In step S813, the CPU 101 performs processing for generating a file, similar to step S609 described above with reference to FIG. Thus, the operation of the image processing apparatus 100 according to the present embodiment is completed.

<Encoding process>
FIG. 9 is a flowchart illustrating an example of the encoding process performed by the image processing apparatus 100 according to the present embodiment. The encoding process shown in FIG. 9 corresponds to step S809 in FIG. 8, and is executed by an instruction from the CPU 101. Here, in the case of the third aspect described above, that is, when the first file including the image group for the selected area 122 and the second file including the image group for the entire image 124 are recorded. Will be described as an example.

  In step S901, the encoding processing unit 114 determines whether or not the imaging unit 112 has changed from the horizontally long state to the vertically long state based on the information set in the register. When a change from the horizontally long state to the vertically long state occurs in the imaging unit 112 (YES in step S901), the process proceeds to step S902. If no change from the horizontally long state to the vertically long state has occurred in the imaging unit 112 (NO in step S901), the process proceeds to step S903.

  In step S902, the encoding processing unit 114 changes a part other than the selection area 122 in the entire image 124 to predetermined data. Thereafter, the process proceeds to step S903. Examples of such predetermined data include a black image. That is, the part other than the selection area 122 in the entire image 124 is painted black, for example.

  In step S903, the encoding processing unit 114 performs image division processing by the image dividing unit 301 in the same manner as in step S701 described above with reference to FIG. Thereafter, the process proceeds to step S904.

  In step S904, the encoding processing unit 114 determines a processing direction. When an image is captured by the horizontally long imaging unit 112, the data corresponding to the upper left pixel in FIG. 5A is the head data of the image data, and the lower right pixel in FIG. Corresponding data is the final data of the image data. In this case, it is not necessary to rotate the image data stored in the memory 102. In this case, the image data held in the memory 102 is set to be encoded in the order of the tile 126A1, the tile 126A2, and the tile 126B, for example. On the other hand, as shown in FIG. 5B, when an image is picked up by the vertically long image pickup unit 112, the data corresponding to the lower left pixel in FIG. 5B is the head data of the image data. The data corresponding to the upper right pixel in 5 (b) is the final data of the image data. In this case, setting is performed such that the image data stored in the memory 102 is rotated 90 degrees to the left. Then, for example, setting is performed on the image data held in the memory 102 such that the encoding process is performed in the order of the tile 126C1, the tile 126C2, the tile 126C3, the tile 126C4, and the tile 126D. Thereafter, the process proceeds to step S905.

  In step S905, the encoding processing unit 114 sets a value in the quantization table set in the code amount control unit 307, that is, a quantization value, as in step S702 described above with reference to FIG. Thereafter, the process proceeds to step S906.

  In step S906, the encoding processing unit 114 performs the encoding process on the entire image 124 in the same manner as in step S703 described above with reference to FIG. Thereafter, the process proceeds to step S907.

  In step S907, the encoding processing unit 114 includes various information, that is, various kinds of information, in the syntax information, similarly to step S704 described above with reference to FIG. Thus, the encoding process in the present embodiment is performed.

  As described above, according to the present embodiment, the mode of image division is appropriately set based on the inclination of the imaging unit 112. For this reason, according to the present embodiment, a vertically long image and a horizontally long image can be accurately and accurately acquired even when the inclination of the imaging unit 112 changes. Therefore, according to the present embodiment, it is possible to provide an image processing apparatus that can obtain a predetermined image and an image of a selected area in a good and accurate manner.

[Modified Embodiment]
As mentioned above, although this invention was explained in full detail based on suitable embodiment, this invention is not limited to these embodiment, Various forms in the range which does not deviate from a summary are also contained in this invention.
For example, in the above embodiment, the case where the selection region 122 is vertically long has been described as an example, but the present invention is not limited to this. For example, a horizontally long selection area 122 may be set. Also in this case, a vertically long image and a horizontally long image can be acquired in parallel.
In the above embodiment, the case where the image processing apparatus 100 includes the imaging unit 112 has been described as an example. However, the image processing apparatus 100 may not include the imaging unit 112. For example, an image input from the outside of the image processing apparatus 100 via an input unit (acquisition unit) may be encoded by the image processing apparatus 100 according to the present embodiment.
In the above embodiment, the case where the image processing apparatus 100 includes the display unit 116 has been described as an example. However, the display unit 116 may be provided separately from the image processing apparatus 100.

  The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100: Image processing apparatus 101: CPU
DESCRIPTION OF SYMBOLS 112 ... Imaging part 114 ... Coding process part 115 ... Display control part 116 ... Display part 117 ... Communication control part 118 ... Communication part 121 ... Inclination detection part 122 ... Selection area 301 ... Image division part

Claims (18)

An acquisition means for acquiring an image;
Setting means for setting a selection region for selecting at least a part of the image;
A dividing unit that divides the image into tiles that are regions that can be decoded without referring to pixel values of other regions based on the setting of the selection region by the setting unit;
An image processing apparatus comprising: encoding means for encoding the tile corresponding to the selection area.   The image processing apparatus according to claim 1, wherein the encoding unit also encodes the tile corresponding to an area other than the selected area. It further has a display control means for displaying the selection area on a display screen,
The selection area is vertically long,
When the display screen is in the horizontally long state, the display control means displays the vertically long selection area on the display screen in the horizontally long state,
3. The image processing according to claim 1, wherein when the display screen is in a vertically long state, the display control unit displays the vertically long selection area on the display screen in a vertically long state. apparatus.   The image processing apparatus according to claim 3, wherein the setting unit sets the size of the selection area so as to fit on the display screen.   The image processing apparatus according to claim 1, wherein the selection area is changeable.   The image processing apparatus according to claim 5, wherein the selection area is capable of at least one of movement, enlargement, and reduction. The acquisition means sequentially acquires a plurality of the images constituting a moving image,
The image processing apparatus according to claim 5, wherein the setting unit changes the selection area when a group of pictures including a plurality of the images is switched.   The image processing apparatus according to claim 1, wherein the selection area is selected from a plurality of candidates.   A mode of division performed by the dividing unit on the image acquired in a state in which the acquiring unit is horizontally long, and a mode in which the dividing unit is performed on the image acquired in a state in which the acquiring unit is vertically long. The image processing apparatus according to claim 1, wherein the division mode is different.   The image processing apparatus according to claim 1, wherein the setting unit enlarges or reduces the selection area in accordance with a change in an angle of view.   The image processing apparatus according to claim 1, wherein the setting unit holds the size of the selection region constant even when the attitude of the acquisition unit is changed. .   The said encoding means uses predetermined data for the said tile corresponding to parts other than the said selection area, when the attitude | position of the said acquisition means changes, The any one of Claim 1 to 11 characterized by the above-mentioned. Image processing apparatus.   The said encoding means scans the said selection area | region in the same direction, without depending on the attitude | position of the said acquisition means when the said image is acquired, The any one of Claim 1 to 12 characterized by the above-mentioned. An image processing apparatus according to 1.   5. The image processing apparatus according to claim 3, wherein the setting unit sets the selection area when the display screen is directed to a subject.   The image processing apparatus according to any one of claims 1 to 13, wherein the setting unit sets the selection area when communication with a predetermined device is established.   A first image group generated by sequentially encoding the tiles corresponding to the selected area; and a second image generated by sequentially encoding the tiles corresponding to an area other than at least the selected area. 16. The image processing apparatus according to claim 1, further comprising file generation means for generating an image group. Setting a selection area for selecting at least a part of the image acquired by the acquisition means;
Dividing the image into tiles that are regions that can be decoded without referring to pixel values of other regions based on the selection region set in the step of setting the selection region;
And a step of encoding the tile corresponding to the selected area. On the computer,
Setting a selection area for selecting at least a part of the image acquired by the acquisition means;
Dividing the image into tiles that are regions that can be decoded without referring to pixel values of other regions based on the selection region set in the step of setting the selection region;
A program for executing the step of encoding the tile corresponding to the selection area.

Images