CN101656833B - image processing device - Google Patents

Abstract

A section specification unit (222) sequentially specifies a motion section D and a non-motion section D 'while changing the motion section D and the non-motion section D'. Each time a motion segment D is designated, a value determination unit (223) determines the average value of the array of image variation values E in the motion segment D as a function value a of an isolated rectangular function r [ x ], and determines the average value of the array of image variation values E in the non-motion segment D' as a function value b of the isolated rectangular function r [ x ]. Each time a motion section D is designated, a deviation degree calculation unit (224) calculates the function values a and b and the deviation degree J of the image change amount array E. Then, only the frame image included in the motion section D designated when the degree of deviation J is minimum is extracted.

Description

image processing device

technical field

The present invention relates to an image processing device, and in particular to a technique for extracting a part of images from a plurality of images.

Background technique

Conventionally, there is known a digital camera that extracts an image of a subject's movement from a plurality of consecutive images in time sequence (refer to Japanese Laid-Open Patent Publication No. 2008-78837). Specifically, when a plurality of images obtained by continuous shooting are sequentially read in chronological order, and the change amount of the currently read image from the previous read image is greater than or equal to a predetermined value, the currently read image is displayed. With such a digital camera, only images corresponding to scenes with large subject changes and large subject movements are extracted.

However, in the conventional digital camera described above, images are unconditionally extracted when the amount of change between temporally adjacent images is equal to or greater than a threshold value. Therefore, when the shooting situation changes, it may not be possible to properly extract the motion of the subject. group of images. For example, even if the subject is still, if hand shake or flickering of fluorescent lights occurs, the amount of change between images will increase. In this case, in the above-mentioned conventional digital camera, regardless of whether the subject itself is still, it is judged that the subject is moving a lot, and an image is extracted.

Contents of the invention

Therefore, the technical problem to be solved by the present invention is to provide an image processing device and a recording medium for accurately extracting a part of images with a large subject movement from a plurality of images regardless of shooting conditions.

According to a first aspect of the present invention, there is provided an image processing device including: an image input unit that inputs a plurality of consecutive images in chronological order; The amount of change between the above adjacent images; the section specifying unit sequentially changes the first section contained in all the sections in which the plurality of images exist in the time order, and sequentially specifies the first section, and at the same time, sequentially specifying a second section that is a section obtained by excluding the sequentially specified first section from all the sections; The first value is determined based on each of the first changes calculated by the change calculation unit, and at the same time, a second value is determined based on each of the second changes calculated by the change calculation unit from among the images existing in the specified second interval. value; the degree of deviation calculation unit calculates the first degree of deviation between each of the first variations and the first value each time the first interval is designated, and at the same time, each time the second interval is designated, Calculate the second degree of deviation of each of the second variation and the second value respectively; the image extraction unit extracts when the sum of the calculated first degree of deviation and each second degree of deviation is the minimum, at this time An image existing in the first section specified by the section specifying unit.

According to a second aspect of the present invention, there is provided an image processing apparatus including an image input unit for inputting a plurality of consecutive images in chronological order; and an extraction number setting unit for setting the number of images to be extracted from the input plurality of images. The change amount calculation unit calculates the change amount between the images specified by the temporally adjacent images among the plurality of input images; the change amount identification unit sequentially specifies the change amount calculated by the change amount calculation unit Among the change amounts, each change amount smaller than the change amount corresponding to the number set by the extraction number setting section; the image deleting section deletes from the input plurality of images specified by the change amount specifying section The extracted images of each change amount; the image extracting unit extracts an image that has not been deleted by the image deleting unit from the input plurality of images.

According to a third aspect of the present invention, there is provided a computer-readable recording medium storing a program for causing the computer to function as follows: an image input unit that inputs a plurality of images that are sequential in time; a variation calculation unit , respectively calculate the amount of change between the temporally adjacent images in the plurality of input images; the interval specifying unit sequentially changes the first one contained in all the intervals in which the plurality of images exist in the temporal order. The first interval is sequentially designated while intervals are performed, and a second interval is sequentially designated as an interval obtained by excluding the sequentially designated first interval from among the above-mentioned all intervals; Among the images existing in the , the first value is determined based on the first change amount calculated by the change amount calculation unit, and at the same time, from each image existing in the specified second interval, the first value is determined based on the change amount calculated by the The second value is determined by each second change amount calculated by the amount calculation unit; the deviation degree calculation unit calculates the first deviation between each first change amount and the first value each time the first section is designated. At the same time, when specifying the second interval each time, calculate the second degree of deviation of each of the second changes and the second value; the image extraction unit extracts the calculated first degree of deviation When the sum of the second degrees of deviation and the respective second degrees of deviation is the minimum, the image existing in the first section designated by the section specifying unit at this time.

According to a fourth aspect of the present invention, there is provided a computer-readable recording medium storing a program for causing the computer to function as follows: an image input unit that inputs a plurality of images that are sequential in time; an extraction number setting unit , setting the number of images to be extracted from the plurality of input images; the change amount calculation unit calculates the amount of change between the images specified by temporally adjacent images among the plurality of input images; The change amount specifying unit sequentially identifies each change amount calculated by the change amount calculation unit that is smaller than the change amount corresponding to the number set by the extraction number setting unit; Deleting an image specifying each amount of change specified by the change amount specifying unit from among the plurality of input images. The image extracting unit extracts an image not deleted by the image deleting unit from the plurality of input images.

According to the present invention, it is possible to accurately extract an image group with a large subject movement from among a plurality of images regardless of imaging conditions.

Description of drawings

FIG. 1 is a hardware configuration diagram of an image processing device according to a first embodiment of the present invention;

2 is a block diagram showing the functional configuration of the image processing device according to the first embodiment of the present invention;

3 is a flowchart of the flow of continuous shooting processing according to the first embodiment of the present invention;

FIG. 4 is a diagram showing an example of the relationship between an imaging range and a subject according to the first embodiment of the present invention;

5 is a diagram for explaining a reduced image arrangement and an image change amount arrangement according to the first embodiment of the present invention;

6 is a flowchart showing the flow of image change amount calculation processing according to the first embodiment of the present invention;

7 is a diagram showing an example of an isolated rectangular function and an array of image change amounts according to the first embodiment of the present invention;

8 is a flowchart of image extraction processing according to the first embodiment of the present invention;

9 is a diagram showing a specific example of an isolated rectangular function according to the first embodiment of the present invention;

10 is a diagram of a specific example of an isolated rectangular function according to the first embodiment of the present invention;

11 is a block diagram showing the functional configuration of an image processing device according to a second embodiment of the present invention;

12 is a flowchart of the flow of image extraction processing according to the second embodiment of the present invention;

13 is a diagram for explaining changes in the reduced image array and the image change amount array in image extraction processing according to the second embodiment of the present invention.

In the picture:

100...image processing device, 1...optical lens device, 2...shutter device, 3...actuator, 4...CMOS sensor, 5...AFE, 6...TG , 7...DRAM, 8...DSP, 9...CPU, 10...RAM, 11...ROM, 12...LCD display controller, 13...LCD display, 14.. .operation section, 15...memory card, 210...image input section, 220...image processing section, 230...operation accepting section, 240...display section, 250...storage section, 260 ...control department

Detailed ways

[First Embodiment]

Next, a first embodiment of the present invention will be described with reference to the drawings.

FIG. 1 is a diagram showing a hardware configuration of an image processing device 100 according to a first embodiment of the present invention. The image processing device 100 can be constituted by, for example, a digital camera.

Image processing device 100 includes optical lens device 1, shutter device 2, actuator 3, CMOS sensor 4, AFE5, TG6, DRAM7, DSP8, CPU9, RAM10, ROM11, liquid crystal display controller 12, liquid crystal display 13, operation part 14 and memory card 15.

The optical lens device 1 is composed of a focus lens, a zoom lens, and the like. The focus lens is a lens for forming a subject image onto the light-receiving surface of the CMOS sensor 4 .

The shutter device 2 functions as a mechanical shutter for blocking the light beam incident on the CMOS sensor 4 and also functions as an aperture for adjusting the light quantity of the light beam incident on the CMOS sensor 4 . The shutter device 2 is constituted by shutter blades and the like. The actuator 3 opens and closes the shutter blades of the shutter device 2 under the control of the CPU 9 .

The CMOS sensor 4 is an imaging sensor that photoelectrically converts (photographs) a subject image incident from the optical lens device 1 . The CMOS sensor 4 photoelectrically converts the subject image at regular intervals based on the clock pulse supplied from the TG6 to accumulate image signals, and sequentially outputs the accumulated image signals. The CMOS sensor 4 is composed of a CMOS (Complementary Metal Oxide Semiconductor) type imaging sensor and the like.

AFE (Analog Front End) 5 performs various signal processing such as A/D (Analog/Digital) conversion processing on the image signal supplied from CMOS sensor 4 based on the clock pulse supplied from TG6, and generates a digital signal and outputs it.

According to the control based on CPU9, TG (Timing Generator) 6 supplies clock pulses to CMOS sensor 4 and AFE5 respectively at regular intervals.

DRAM (Dynamic Random Access Memory) 7 temporarily stores digital signals generated by AFE5 and image data generated by DSP8.

DSP (Digital Signal Processor) 8 performs various image processing such as white balance correction processing, γ correction processing, and YC conversion processing on the digital signal stored in DRAM 7 under the control of CPU 9 to generate a signal composed of luminance signals and color difference signals. frame image data. In the following description, an image represented by this frame image data is referred to as a frame image.

A CPU (Central processing Unit) 9 controls the overall operation of the image processing device 100 . The RAM (Random Access Memory) 10 functions as a work area when the CPU 9 executes each process. A ROM (Read Only Memory) 11 stores programs and data necessary for the image processing device 100 to execute each process. CPU9 uses RAM10 as a work area, and executes each process by cooperating with the program memorize|stored in ROM11.

The liquid crystal display controller 12 converts the frame image data stored in the DRAM 7 and the memory card 15 into an analog signal and outputs it under the control of the CPU 9 . The liquid crystal display 13 displays images and the like expressed by analog signals supplied from the liquid crystal display controller 12 .

The operation unit 14 accepts operations of various buttons from the user. The operation unit 14 has a power button, a cross button, a determination button, a menu button, a shutter button, and the like. The operation unit 14 supplies signals corresponding to various button operations received from the user to the CPU 9 . After receiving these signals from the operation unit 14, the CPU 9 executes processing based on the received signals.

The memory card 15 is a recording medium for recording frame image data generated by the DSP 8 .

FIG. 2 is a block diagram showing the functional configuration of the image processing device 100 according to the present embodiment. In this embodiment, it is assumed that the image processing device 100 includes an image input unit 210 , an image processing unit 220 , an operation accepting unit 230 , a display unit 240 , a recording unit 250 , and a control unit 260 .

The image input unit 210 inputs a plurality of frames of image data under the control of the control unit 260 . Frame images respectively represented by the plurality of frame image data are consecutive in time order. The image input unit 210 can be realized by the optical lens device 1, shutter device 2, actuator 3, CMOS sensor 4, AFE5, TG6, DRAM7 and DSP8 shown in FIG.

The image processing unit 220 executes image change amount calculation processing and image extraction processing described later under the control of the control unit 260 . The image processing unit 220 includes an image change amount calculation unit 221 , an interval designation unit 222 , a value determination unit 223 , a degree of deviation calculation unit 224 , and an image extraction unit 225 .

The image change amount calculating unit 221 calculates the amount of change (for example, the sum of differences in pixel values) between the reduced images adjacent in time order from the reduced images of the frame images input from the image input unit 210 . The image change calculation unit 221 can be realized by the CPU 9 shown in FIG. 1 .

The section specifying section 222 sequentially changes a predetermined motion section D (first section) in all sections in time order in which the reduced images of all the frame images input from the image input section 210 exist, and simultaneously specifies the changed motion section D sequentially. . The section specifying unit 222 sequentially specifies a non-moving section (second section) as a section obtained by excluding the moving section D from the above-mentioned all sections. The motion interval D and the non-motion interval D' will be described later. The section specifying unit 222 supplies the value determining unit 223 with the result of specifying the moving section D and the non-moving section D'. The section specifying unit 222 can be realized by the CPU 9 shown in FIG. 1 . The motion section D may be, for example, an area in which there is no gap in a reduced frame image, for example, in all sections, and the non-motion section D' may be a section obtained by excluding the motion section D from all sections.

The value determining unit 223 determines a value based on the amount of change between the reduced images existing in the motion segment D specified by the segment specifying unit 222 (for example, the average value of the amount of change between the reduced images existing in the motion segment D). is the first value. In addition, the value determination unit 223 sets a value based on the amount of change between the reduced images existing in the non-moving period D′ specified by the period specifying unit 222 (for example, between the reduced images existing in the non-moving image period D′). The average value of the amount of change) is determined as the second value. The value determination unit 223 supplies the determination results of the first value and the second value to the degree of deviation calculation unit 224 . The value determination unit 223 can be realized by the CPU 9 shown in FIG. 1 .

The degree of deviation calculation unit 224 calculates the degree of deviation (hereinafter referred to as the first value) of each first change amount and the first value (the average value of each first change amount, etc.) each time the motion interval D is designated by the interval designation unit 222. 1 degree of deviation). In addition, the degree of deviation calculation unit 224 calculates the degree of deviation ( Hereinafter, it is referred to as the second degree of deviation). The deviation calculation unit 224 can be realized by the CPU 9 shown in FIG. 1 .

The image extracting unit 225 extracts from the reduced image of the frame image input from the image input unit 210 when the sum of the first degree of deviation and the second degree of deviation calculated by the degree of deviation computing unit 224 is the minimum, and at this time is specified by the section. The reduced image existing in the motion section D specified by the unit 222. The image extraction unit 225 can be realized by the CPU 9 shown in FIG. 1 .

The operation accepting unit 230 accepts a user's operation on the image processing device 100 . As this operation, there are an operation to instruct the image processing device 100 to be photographed by the user, an operation to set the number of images taken by the user (hereinafter referred to as the number of photographed images), and an operation to set the number of frame images extracted by the image extracting unit 225 (hereinafter referred to as the number of frames). to extract the number of sheets) and so on. The operation accepting unit 230 supplies the control unit 260 with a result of accepting the user's operation. The operation acceptance unit 230 can be realized by the operation unit 14 shown in FIG. 1 .

The display unit 240 displays frame images and the like input by the image input unit 210 . The display unit 240 can be realized by the liquid crystal display controller 12 and the liquid crystal display 13 shown in FIG. 1 .

The recording unit 250 records frame image data expressing the frame image extracted by the image extracting unit 225 . The recording unit 250 can be realized by the memory card 15 shown in FIG. 1 .

The control unit 260 collectively controls the processing performed by each unit. Control unit 260 can be realized by CPU9, RAM10, and ROM11 shown in FIG. 1 .

FIG. 3 is a flowchart showing an example of the flow of continuous shooting processing executed by the image processing device 100 . This continuous shooting processing will be described as shooting processing executed by the control unit 260 (CPU 9 ). In addition, the continuous shooting process starts when the user performs a predetermined operation on the operation accepting unit 230 .

The control unit 260 starts the continuous shooting process and simultaneously supplies the frame images input from the image input unit 210 to the display unit 240 sequentially, thereby displaying the frame images on the display unit 240 as a live preview image.

In step S1 , the control unit 260 determines whether or not the user has performed an operation to set the number of images to be captured and the number of images to be extracted. Specifically, the control unit 260 judges whether the operation of setting the number of images to be captured and the number of images to be extracted is performed by the user based on whether a signal corresponding to the operation of setting the number of images to be captured and the number of images to be extracted is supplied from the operation accepting unit 230 . When the determination in step S1 is YES, the control unit 260 advances the process to step S2 after setting the number of photographed sheets and the number of extracted sheets corresponding to the user's operation. On the other hand, when the determination in step S1 is negative, the control unit 260 returns to the processing in step S1.

In the following description, it is assumed that N sheets are set as the number of photographed sheets, and M sheets are assumed as the number of extracted sheets. That is, M frame images in which the subject motion is large between temporally successive images are extracted from N temporally continuous frame images. For example, as shown in FIG. 4, the image processing device 100 continuously captures N images of a car passing through the shooting range of the CMOS sensor 4, and extracts only the images of the moving car within the shooting range from the N consecutive images. M images.

In step S2 , the control unit 260 monitors a signal corresponding to the operation of the shutter button supplied from the operation accepting unit 230 . After detecting a signal corresponding to the user's operation of the shutter button, the control unit 260 inputs N frame images consecutive in time order to the image input unit 210 (continuous shooting). In the following description, these N frame images are represented as p[x] (0≤x≤N-1), respectively. x is the index number added on each frame image. Here, index numbers 0, 1, 2, . . . , N−1 are sequentially added from the earliest frame image in time order. An arrangement in which frame images are arranged in index order, that is, in time order, is referred to as an image arrangement P.

In step S3 , the control unit 260 reduces the captured N frames of images through an image processing unit not shown in the figure, and generates N reduced images arranged in time order. This reduction processing is processing for reducing the number of pixels of an image that is generally performed. The reduction ratio can be appropriately determined according to the characteristics of the camera in consideration of the minimum size of the subject and the influence of hand shake. In the following description, the N reduced images are represented as ps[x] (0≤x≤N-1), respectively. In this reduction process, the index number of the frame image is made to match the index number of the reduced image generated from the frame image. That is, similarly to frame images, index numbers 0, 1, 2, .

As shown in FIG. 5 , the arrangement in which these reduced images are arranged in the order of index numbers, that is, in time order, is set as the reduced image arrangement PS.

In step S4, the control unit 260 performs image change amount calculation processing. That is, as shown in FIG. 5 , with respect to the reduced image sequence PS, the amount of change between the reduced images ps[x] successive in time is calculated as the image change amount e. The index number of the image change amount e is referred to as the above-mentioned x, and each calculated image change amount is expressed as e[x] (0≤x≤N-2). Here, index numbers of 0, 1, 2, . The sequence in which these image change amounts e[x] are arranged in order of index numbers, that is, in time order, is set as an image change amount sequence E. Details of the image change amount calculation processing in step S4 will be described later.

In step S5 , the control unit 260 extracts only the reduced image with relatively large subject image motion from the N reduced images. Details of the image extraction process in step S5 will be described later.

In step S6, the control unit 260 adjusts the reduced images extracted by the process of step S5 into M images. This is because the number of extracted sheets set by the user's operation is M sheets, and when the reduced images of more than the M sheets are extracted by the processing of step S5, fewer than M sheets are extracted by the processing of step S5. In the case of reduced images of the number of reduced images, it is necessary to adjust the number of reduced images to M desired by the user.

Thus, specifically, when the number of extracted reduced images is greater than M, for example, images included in the motion section D are extracted at predetermined intervals, and the number of images is M. On the other hand, when the extracted reduced images are less than M, for example, by expanding the range of the moving interval D, the number of reduced images included in the moving interval D is set to M.

In step S7 , the control unit 260 records in the recording unit 250 only the frame image data corresponding to the reduced image whose number of extracted images has been adjusted by the process of step S6 . At this time, the control unit 260 temporarily displays on the display unit 240 a synthesized image obtained by combining the image data of each frame recorded on the recording unit 250 and a reduced image of the image data of each frame.

FIG. 6 is a flowchart showing an example of a detailed flow of image change amount calculation processing in step S4. Details of the image change amount calculation processing will be described with reference to FIG. 6 . In the following description, this image change amount calculation process is performed by the image processing unit 220 under the control of the control unit 260 .

This image change amount calculation process is a process of calculating the sum of the absolute value differences of the pixel values of pixels located at the same position among the reduced images ps adjacent to each other in time order as the image change amount e. Therefore, the image change amount e is specified by the reduced images ps adjacent to each other in time order.

In the description of this image change amount calculation process, the provisional value of the sum of the image changes most e is assumed to be d. For convenience, the index number of the reduced image is represented by i instead of the above-mentioned x. As for the number of pixels of the reduced image ps[i], p is the number of pixels in the horizontal direction, that is, the x direction, and q is the number of pixels in the vertical direction, that is, the y direction. Also, the position of an arbitrary pixel on the reduced image ps[i] is represented by coordinates (x, y).

The image processing section 220 sets 0 (zero) as the index number i (step s11), sets the initial value of the tentative value d to 0 (step S12), sets the initial value of the y coordinate to 1 (step S13), and sets The initial value of the x coordinate is set to 1 (step S14).

Next, the image change amount calculation unit 221 of the image processing unit 220 calculates the pixel value of the pixel on the coordinate (x, y) of the reduced image ps[i] and the pixel value of the pixel on the coordinate (x, y) of the reduced image ps[i+1]. The absolute value of the difference between the pixel values of the pixels of , and the calculated absolute value of the difference is added to the provisional value d (step S15).

Next, the image processing unit 220 judges whether x is p (step 316). When the determination in step S16 is YES, the image processing unit 220 advances the process to step S18. On the other hand, when the determination in step S16 is NO, the image processing unit 220 increments the coordinate x (adds 1 to x) (step S17 ), and returns the process to step S15 .

Next, the image processing unit 220 judges whether y is q (step S18). When the determination in step S18 is negative, the image processing unit 220 increments the coordinate y (adds 1 to y) (step S19 ), and returns the process to step S14 .

On the other hand, when the determination in step S18 is YES, the image processing unit 220 holds the image change amount e[i] as a tentative value d (step S20 ). Next, the image processing unit 220 judges whether or not the current index number i is N-2 (N is the number of shots set in the process of step S1) (step S21). When the determination in step S21 is NO, the image processing unit 220 increments the counter value i (increments i by 1) (step S22), and returns the process to step S12. On the other hand, when the determination in step S21 is YES, the image processing unit 220 terminates the image change amount calculation process.

Next, the image extraction process in step S5 will be described with reference to FIG. 7 . In FIG. 7 , the horizontal axis represents the index number x, and the vertical axis represents the image change amount e[x]. In the figure, the curve indicated by the solid line represents the image change amount e[x] corresponding to each index number x. In addition, the curve indicated by the dotted line represents the isolated rectangular function r[x] described later. In addition, since the value of the image change amount e[x] is a discrete value defined for each index number which is an integer value, it is shown in FIG. 7 as each value of the image change amount e[x] connected by a straight line.

First, let the index numbers corresponding to the start point and end point of the motion section D be x1 and x2, respectively. Accordingly, the motion section D is expressed as a section from x1 to x2 as follows.

D=[x1, x2] (0<x1<x2<N-2)

Here, [x1, x2] represents the interval from x1 to x2. These x1, x2 are the separation points (boundaries) between the motion section D and the non-motion section D'. Then, the section specifying section 222 of the image processing section 220 sequentially specifies all combinations of the above-mentioned start point x1 and end point x2, and assigns a motion section D and a plurality of non-motion sections D respectively defined by a plurality of combinations of these x1, x2. Carry out the following processing.

The value determination unit 223 of the image processing unit 220 determines, for each combination of x1 and x2 specified by the section specifying unit 222, based on the image change amounts e[x] corresponding to all index numbers x included in the motion section D, 1st value. The value determination unit 223 of the image processing unit 220, for each combination of x1 and x2 specified by the section specifying section 222, based on the amount of image change e[x] corresponding to all index numbers x included in the non-moving section D′ , to determine the second value. Specifically, the value determination unit 223 calculates a which is an average value of each image change amount e[x] of each image change amount array E belonging to the motion section D, and determines this a as the first value. The value determination unit 223 calculates b which is the average value of each image change amount e[x] of each image change amount array E belonging to the non-moving interval D', and determines this b as the second value.

The degree of deviation calculation unit 224 of the image processing unit 220 defines the isolated rectangular function r[x] as follows for each combination of x1 and x2 specified by the section specifying unit 222 .

$\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}\mathrm{<span\; class="notranslate">\; a</span>}& <span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; \&NotElement;</span>\mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; )</span>\\ \mathrm{<span\; class="notranslate">\; b</span>}& <span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; )</span>\end{array}\right.$

The value of this isolated rectangular function r[x] is a in the moving interval D, and b in the non-moving interval D'.

The degree of deviation calculating unit 224 of the image processing unit 220 calculates the degree of deviation between the isolated rectangular function r[x] and the image variation array E for each combination of x1 and x2 specified by the section specifying unit 222 . The so-called degree of deviation J refers to the sum of the difference between the isolated rectangular function r[x] of each index number x and the image change amount arrangement E. Specifically, the sum of squared differences between the isolated rectangle shape function r[x] and the image change amount array E is set as the degree of deviation J as in the following equation.

$\mathrm{<span\; class="notranslate">\; J</span>}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; e</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}$

If the motion interval defined by x1 and x2 designated by the interval specifying unit 222 is called the motion interval Dmin when the degree of deviation J is the minimum, the image extracting unit 225 extracts from the reduced image of the frame image input by the image input unit 210. A reduced image ps which prescribes all image changes e included in the movement interval Dmin, respectively.

Next, a schematic flow of the image extraction process in step S5 will be described. In the following description, set the minimum value of the deviation degree J as Jmin, set the starting point of the motion interval D of the deviation degree Jmin as m1, set the ending point as m2, and set the corresponding starting point m1 and ending point m2 The index numbers of the reduced image array PS are xin and xout.

First, the image processing unit 220 sequentially calculates the degree of deviation J while changing x1 and x2, and compares the degrees of deviation J calculated sequentially with each other to obtain the degree of deviation Jmin which is the minimum value among the degrees of deviation J calculated sequentially. Then, the image processing unit 220 obtains the start point m1 and the end point m2 of the motion section Dmin specified when the degree of deviation J becomes the minimum value Jmin. At this time, the image processing unit 220 performs a complete search for all possible combinations of x1 and x2. Here, x1, x2 are discrete values.

The so-called complete search in the first embodiment refers to changing x1 between 0 and (N-3), while changing x2 between x1 and (N-2), and checking the degree of deviation J for all combinations of x1 and x2. deal with. Specifically, first, x1 is fixed at 0, and x2 is changed between 1 and (N-2). Next, fix x1 to 1, and change x2 between 2 and (N-2). Next, x1 is fixed at 2, and x2 is changed between 3 and (N-2). In this way, the image processing unit 220 sequentially increments the value of x1 from 0 to (N−3), and simultaneously increments x2 sequentially corresponding to each x1. The image processing unit 220 checks the degree of deviation J calculated this time every time x1 or x2 changes by 1.

Usually, since the number of frame images obtained by one continuous shooting is about several, x which is an index of frame images is relatively limited. Therefore, the processing load by the complete search on the control unit 260 (CPU 9 ) is small.

FIG. 8 is a flowchart showing the detailed flow of the image extraction process in step S5. Details of the image extraction process will be described with reference to FIG. 8 . In the following description, this image extraction process is performed by the image processing unit 220 under the control of the control unit 260 .

First, the image processing unit 220 sets the initial value of Jmin to a predetermined value close to infinity (step S31). Next, the section specifying unit 222 of the image processing unit 220 sets the initial value of x1 to zero (step S32 ), and sets the initial value of x2 to x1+1 (step S33 ). Next, the value determination unit 223 of the image processing unit 220 determines the function value a of the isolated rectangular function r[x] in the moving interval D and the function value b in the non-moving interval D' (step S34). Next, the degree of deviation calculation unit 224 of the image processing unit 220 calculates the degree of deviation J (step S35 ).

Next, the image processing unit 220 judges whether or not the degree of deviation J calculated by the process of step S35 is smaller than Jmin (step S36 ). If the determination in step S36 is negative, the image processing unit 220 advances the processing to step S39. On the other hand, when the determination in step S36 is YES, the image processing unit 220 advances the process to step S37. In step S37, the image processing unit 220 sets the degree of deviation J calculated by the processing in step S35 as Jmin. Next, the image processing unit 220 temporarily sets the start point m1 to x1, and temporarily sets the end point m2 to x2 (step S38). '

In step S39, the image processing unit 220 judges whether x2 is N-2 (N is the number of shots set in the process of step S1). When the determination in step S39 is YES, the image processing unit 220 advances the processing to step S41. Next, the image processing unit 220 judges whether or not x1 is N-3 (step S41). When the determination in step S41 is YES, the image processing unit 220 advances the process to step S43. When the determination in step S41 is NO, the image processing unit 220 increments the count value x1 (increments x1 by 1) (step S42), and returns the process to step S33.

On the other hand, when the determination in step S39 is NO, the image processing unit 220 increments the counter value x2 (increments x2 by 1) (step S40), and returns the process to step S34.

In step S43, since the loop processing of steps S33-S41 is terminated, the start point m1 and the end point m2 are determined. These m1 and m2 are index numbers of the image change amount array E, and the image change amounts e[m1] and e[m2] of m1 and m2 are respectively calculated from the two reduced images ps. Therefore, it is necessary to determine which of the two reduced images PS to use as ps[xin] and ps[xout] corresponding to e[m1] and e[m2]. Therefore, in step S43, the image processing unit 220 sets (m1+1) as the start point xin, and sets m2 as the end point xout.

In step S44 , in the reduced image array PS, a series of time-sequential reduced images are extracted from the reduced image ps[xin] at the start point to the reduced image ps[xout] at the end point of the motion section D. Furthermore, the image processing unit 220 terminates the image extraction process after the process of step S44.

In the first embodiment, an isolated rectangular function r[x] having a and b as function values is defined as a comparison object of the image change amount array E. Since the function value a is the average value of the image change amount arrangement E in the motion interval D, and the function value b is the average value of the image change amount array E in the non-motion interval D', so according to the starting point x1 and the end point of the motion interval D The position of x2 is changed dynamically. Therefore, only by comparing the threshold values a, b and the image change amount arrangement E, the minimum value Jmin of the deviation degree J can be obtained, so that the image change amount e[m1] of the starting point m1 and the end point m2 of the deviation degree Jmin can be obtained. The image change amount e[m2] of is reliably extracted as a relatively large image change amount. As a result, a group of frame images with a large amount of image change, that is, a group of frame images with a large subject motion can be extracted.

Fig. 9(a) - (d) are specific examples of the isolated rectangular function r[x] in the case of x1 = 1, and Fig. 10 (a) - (d) are the isolated rectangular function r[ in the case of x1 = 3 x] for a specific example. In FIGS. 9( a ) to ( d ) and FIGS. 10( a ) to ( d ), the horizontal axis represents the index number x, and the vertical axis represents the amount of image change. In Figure 9(a)-(d) and Figure 10(a)-(d), the rectangular wave drawn with thick solid line represents the isolated rectangular function r[x], and the waveform drawn with thin solid line represents the arrangement of image changes e. In addition, since the value of the image change amount e[x] is a discrete value, in these Figs. values of .

As shown in these Figures 9 (a) to (d) and Figures 9 (a) to (d), it can be seen that the function value a of the isolated rectangular function r[x] is the average value of the image change amount array E in the motion interval D, The function value b is the average value of the image variation arrangement E in the non-moving interval D', so a and b change dynamically according to the positions of the starting point x1 and the ending point x2 of the moving interval D. Since the waveform representing the isolated rectangular function r[x] in Fig. 10(c) is the closest to the waveform representing the arrangement E of the image variation, it can be understood that the degree of deviation J is at x1=3, x2=6 in Fig. 10(c) case minimum.

As described above, the image processing device 100 according to the first embodiment takes the function value a of the isolated rectangular function r[x] as the average value of the image change amount array E in the moving section D, and takes the function value b as the non-moving section D' The average value of the image variation ranking E in the medium. Next, Jmin at which the degree of deviation J between the function values a, b and the image variation array E is the smallest is obtained. Then, the image p[xin] corresponding to the image change amount e[m1] of the start point m1 under the deviation degree Jmin and the image p[xout] corresponding to the image change amount e[m2] of the end point m2 are obtained. Then, from the image arrangement P, temporally continuous frame image groups from the frame image p[xin] to the frame image P[xout] are extracted.

In this way, the two function values a and b are not constant values, but the average value of the image variation array E according to the dynamically changing positions of the starting point x1 and the ending point x2 of the motion interval D. Therefore, only by comparing the threshold values a, b and the image change amount arrangement E, and finding the minimum value Jmin of the deviation degree J, the image change amount e[m1] of the starting point m1 under the deviation degree Jmin, the end point The image change amount e[m2] of the point m2 is reliably extracted as a relatively large image change amount. Therefore, it is possible to accurately extract a group of frame images in which the subject motion is large from the temporally continuous image sequence P. Therefore, the image processing device 100 of the present embodiment can, for example, only start from the continuous shooting of such subjects (track and field players and balls) as passing through the scene within the shooting range, such as track and field competitions and ball games, and the continuous shooting of images such as gymnastics competitions. An image in which the subject (gymnast) moves in the order of stillness, movement, and stillness within the shooting range is appropriately extracted from an image in which the subject (gymnast) moves greatly within the shooting range.

Also, for example, even if the value of each image change amount e changes due to hand shake, that is, even if the waveform position representing the image change amount array E shown in FIGS. 9 and 10 changes in the up-down direction due to hand shake, the function The values a and b are the average values of the image variation array E, respectively, so the function values a and b also follow this variation. Therefore, the image processing device 100 according to the first embodiment does not require adjustment of the conventional threshold value, is hardly affected by external disturbances such as hand shake, and can appropriately extract frame image groups with large subject movements regardless of the shooting state. , high versatility.

When the background area and the subject area of the image have substantially the same appearance, the amount of change between temporally successive images increases in proportion to the magnitude of the subject's motion between frame images. However, in practice, there are differences in the background area covered by the subject, the difference in the area and content of the subject area, and the difference in the overall edge caused by hand shake between the frame images. The problem of unstable change in the amount of change between frame images that are successive in time.

Therefore, the image processing device 100 of the first embodiment defines an isolated rectangular function r[x] composed of two function values a and b. The isolated rectangular function r[x] only needs to determine the two parameters of the start point x1 and the end point x2 of the motion interval D, and in the motion interval D, the average value a of the image variation e contained in the motion interval D is defined as a function value, and for the non-moving interval D', the average value b of the image variation e included in the non-moving interval D' is defined as the function value. Thus, in the image processing device 100 of the first embodiment, even if some image change amounts e are slightly changed due to noise, the relative relationship between the isolated rectangular shape function r[x] and the image change amount array E is not greatly affected by this effect. Influenced by changes in the amount of image change e, it is strong against external disturbances.

However, in the case of continuous shooting, if the hand shake occurs only for a certain period of time immediately after the shutter button is pressed due to the user pressing the shutter button, the image captured immediately after the shutter button is pressed The amount of change becomes larger. In this case, for example, in the shooting situation shown in FIG. 4 , in two different shooting intervals, namely, the shooting interval immediately after the shutter button is pressed and the shooting interval in which the car passes through the shooting range, temporally consecutive frames The amount of change between images becomes larger. Therefore, in this case, attention should be paid to extracting frame image groups from two different imaging sections. For example, in the imaging situation shown in FIG. 4 , there is a concern that an image captured immediately after the shutter button is pressed, that is, a group of frame images in which the subject does not exist in the imaging range and lacks a change, may be extracted.

However, according to the image processing device 100 of the first embodiment, since only one imaging section (motion section D) for extracting a frame image group is determined, the possibility of extracting an unnecessary frame image group can be reduced. That is, according to the image processing apparatus 100 of the first embodiment, even if there are many peak sections in the waveform representing the image change amount array E due to disturbances such as hand shake, that is, even if there are many peak sections in the image change amount array E, A section with a large amount of image change e can also be excluded from the extraction target, and a section with a large amount of change between images due to the influence of disturbance such as hand shake can be excluded, and an image with a large motion of the subject image can be extracted more reliably.

The image processing device 100 of the first embodiment calculates the image change amount e from the difference between the reduced images that are temporally successive in the reduced image array PS. Since the shooting time interval between the frame images corresponding to the reduced images ps successively in time in the reduced image array PS is short, even if hand shaking, flickering of fluorescent lights, etc. occur, the images of the reduced images ps successively in time The amount of change e does not vary greatly, so it is hardly affected by disturbance. Accordingly, even when the user holds the image processing device 100 with his or her own hand and performs imaging, it is not necessary to perform position matching processing between the reduced images ps, so that the calculation load of the image processing unit 220 (CPU 9 ) can be reduced.

[Second Embodiment]

Next, a second embodiment of the present invention will be described. In the second embodiment, the configuration of the image processing section 220 and the image extraction process in step S5 are different from those in the first embodiment. Since other configurations and processes are the same as those of the first embodiment, description thereof will be omitted.

FIG. 11 is a block diagram showing a functional configuration of an image processing device 100 according to the second embodiment. The functional configuration of the image processing device 100 of the second embodiment differs from that of the first embodiment only in the configuration of the image processing unit 220 . The image processing unit 220 of the second embodiment is assumed to include an image change calculation unit 221 , an extraction number setting unit 226 , an image change identification unit 227 , a judgment unit 228 , an image deletion unit 229 , and an image extraction unit 225 .

In the second embodiment, the image change amount calculating unit 221 first calculates the amount of change (for example, the sum of pixel value differences) between the reduced images successively in time from the reduced images of the frame images input through the image input unit 210 . . And, every time the reduced image is deleted by the image deleting unit 220, for each reduced image obtained by removing the deleted reduced image from the reduced image of the frame image input by the image input unit 210, the reduced images successively calculated in time are repeatedly calculated. the amount of change between.

In response to the user's operation of setting the number of sheets to be drawn, the number of sheets to be drawn setting unit 226 sets the number of sheets to be drawn M corresponding to the operation. In this setting process, a signal indicating the number of sheets to be extracted M set by an operation performed by the extraction user is supplied from the operation accepting portion 230 to the number of sheets to be extracted to the setting portion 226 via the control portion 260 . The drawing number setting unit 226 can be realized by the CPU 9 shown in FIG. 1 .

The image change amount specifying unit 227 specifies the minimum image change amount among the calculated image change amounts each time the image change amount is calculated by the image change amount calculating unit 221 after the reduced image is deleted by the image deleting unit 229 . In addition, at this time, the image change amount specifying unit 227 also specifies two image change amounts that are temporally successive to each other with respect to the specified minimum image change amount. The image change amount specifying unit 227 can be realized by the CPU 9 shown in FIG. 1 .

The judging unit 228 judges, with respect to the smallest image change amount, which of two image change amounts adjacent in chronological order is smaller each time the image change amount specifying unit 227 specifies each image change amount. The determination unit 228 supplies the determination result to the image deletion unit 229 . The judging unit 228 can be realized by the CPU 9 shown in FIG. 1 .

The image deletion unit 229 deletes, from the reduced image array PS, the amount of image change determined to be smaller, indicated by the determination result supplied from the determination unit 228 . The image deletion unit 229 can be realized by the CPU 9 shown in FIG. 1 .

In the second embodiment, the image extracting unit 225 extracts, from the reduced image of the frame image input by the image input unit 210 , the remaining reduced images that have not been deleted by the image deleting unit 229 .

FIG. 12 is a flowchart showing an example of the flow of image extraction processing (processing in step S5 ) in the second embodiment. Details of the image extraction process in the second embodiment will be described with reference to FIG. 12 . In the following description, this image extraction process is performed by the image processing unit 220 under the control of the control unit 260 .

First, in step S51 , the image change amount identification unit 227 of the image processing unit 220 specifies the smallest image change amount e from the image change amount array E. In the following description, the minimum image change amount specified by the process of step S51 is set to e[k], that is, the index number corresponding to the specified minimum image change amount is set to k.

In step S52 , the image change amount specifying unit 227 of the image processing unit 220 specifies the image change amount e[k−1] and the image change amount e[k+1]. The image change amount e[k−1] is an image change amount that is temporally preceding (in the past) adjacent to the image change amount e[k] in the image change amount array P. The image change amount e[k+1] is an image change amount adjacent to the image change amount e[k] in the time order after (in the future) in the image change amount array P.

In step S53 , the judgment unit 228 of the image processing unit 220 judges whether or not the image change amount e[k−1] is smaller than the image change amount e[k+1]. When the determination in step S53 is YES, the image processing unit 220 advances the process to step S54. On the other hand, when the determination in step S53 is NO, the image processing unit 220 advances the process to step S55.

In step S54, the image deletion unit 229 of the image processing unit 220 deletes the two reduced images ps[k] and ps[k+1] corresponding to the image change amount e[k] from the reduced image array PS in chronological order. The reduced image ps[k] located in the front. In other words, the image deletion unit 229 deletes ps[k-1], ps[k], which is located later in time order, among the two reduced images ps[k-1] and ps[k] with the predetermined amount of image change e[k-1]. ].

In step S55, the image deletion unit 229 of the image processing unit 220 deletes the two reduced images ps[k] and ps[k+⊥] corresponding to the image change amount e[k] from the reduced image array PS in chronological order. The downscaled image ps[k+1] located in the back. In other words, the image deletion unit 229 deletes Ps [ k+1].

In step S56, the judging unit 228 of the image processing unit 220 judges whether or not the number of shrunken images ps left in the shrunken image array PS is M as a result of deleting the shrunken images in the preceding step S54 or S55. If the determination in step S56 is negative, the image processing unit 220 advances the process to step S58.

In step S58 , the image deletion unit 229 of the image processing unit 220 deletes two image change amounts e respectively specified by the reduced image ps deleted in the preceding step S54 or step S55 from the image change amount array E. In this deletion process, for example, as shown in FIG. 13 , when the image deletion unit 229 deletes the reduced image ps[k] this time, two images specified by the deleted reduced image ps[k] are deleted. Image change amount e[k], e[k-1], and calculate the change amount of the reduced image ps[k-1], ps[k+1] adjacent to the deleted reduced image ps[k] The image variation e[k-1]' is inserted into the image variation array E.

On the other hand, when the determination in step S56 is YES, the image extraction unit 225 of the image processing unit 220 extracts the remaining M reduced images ps (step S57 ). After the processing in step S57, the image processing unit 220 terminates the image extraction processing.

As above, the image processing unit 220 continues to extract the minimum image change amount in step S51 until the remaining reduced image ps is M pieces (yes in step S56), continuously deletes the reduced image ps in steps S54 and S55, and then In step S58, the image variation e is continuously deleted. And, the smallest image change amount e in the finally remaining image change amount array E that is not deleted in step S58 changes according to the set M value. That is, if the set M value is small, there is no deletion in step S58 and the minimum image change amount e in the final remaining image change amount array E is large. On the other hand, if the set value of M is large, there is no deletion in step S58 and the smallest image change amount e in the final remaining image change amount array E is small. Thus, in step S51, until the remaining reduced images ps are M sheets (yes in step S56), all deletion ratios corresponding to the set M value of the specific image change amount (not deleted in step S58 and finally The smallest image change amount e in the remaining image change amount arrangement E) is the small image change amount e.

According to this embodiment, the image processing apparatus 100 of the second embodiment arranges the image change amounts e in chronological order as the image change amounts E, and compares these image change amounts e with each other to extract an image change amount of a relatively small value. e[k], the reduced image p corresponding to the extracted image change amount e[k] is deleted, and the frame images corresponding to the remaining reduced images in the image array P are extracted. That is, the image processing device 100 according to the second embodiment focuses on the relative relationship between the image change amounts e, and extracts the image p corresponding to the relatively large image change amount e from the image change amount array E. As a result, even when the amount of change e between images changes due to hand shake by the user, the change can be tracked, so that only images with large subject movements can be extracted with high precision regardless of imaging conditions.

The image processing device 100 according to the second embodiment focuses on the relative relationship between the image change amounts e, and deletes only the image p whose image change amount e is relatively small, that is, a redundant image lacking in change, from the image change amount array E. Unlike the extraction of continuous image groups as in the first embodiment, only images with large motion are discretely extracted. Therefore, it is not limited to a specific motion mode as in the first embodiment, but can be applied to a general motion scene in which the subject stays in the image or the subject moves relatively irregularly. Thereby, for example, even in the case of synthesizing captured images of a scene in which the subject irregularly repeats stillness and movement within the photographing range to generate one composite image, it is possible to prevent only redundant images lacking changes from being composited. That is, if a synthesized image is generated by synthesizing the group of frame images extracted by the image processing device 100 of the second embodiment, a synthesized image in which the motion of the subject is relaxed can be obtained.

[deformation, etc.]

The present invention is not limited to the foregoing embodiments, and modifications, improvements, and the like within the scope of achieving the object of the present invention are included in the present invention.

For example, in the first embodiment described above, the degree of deviation J is calculated for all combinations of x1 and x2. However, the present invention is not limited thereto. Even in the case where a<b is the result of calculating the function values a and b of the isolated rectangular function r[x], as long as the difference between the function value b and the function value a is small, it can be obtained from Exclude this combination from your search. Based on this, the combination of x1 and x2 which is not an optimal solution can be clearly excluded, the load required for the calculation processing of the image processing unit 220 (CPU 9 ) can be reduced, and the processing speed can be increased.

In the above-mentioned first embodiment, it is set so that 0<x1<x2<N-2, and x1 and x2 representing the peak portion of the isolated rectangular function r[x] are not 0 or (N-2 ), but it is not limited to this, it can also be set as 0≤x1<x2≤N-2, so that x1 and x2 representing the part of the peak of the isolated rectangular function r[x] can be taken as 0 or (N-2) as the boundary .

In the above-mentioned first embodiment, the shape of the peak portion of the isolated rectangular function may be quadrilateral, but the shape of the peak portion may also be trapezoidal.

In addition, in each of the above-described embodiments, the sum of the absolute values of the differences in pixel values at the same position is calculated as the image change amount e for the reduced images ps that are temporally successive. However, the sum of the squares of pixel value differences at the same position may be calculated as the image change amount e, and any method may be used as long as the difference in the image can be quantified. Also, when the reduced image ps is a color image, a difference may be calculated for each color component of the reduced image ps.

In addition, in each of the above-described embodiments, the difference in pixel values is calculated covering the entire image, but the present invention is not limited to this, and a window or detection frame may be provided to calculate the difference in pixel values only for a part of the image. Thus, since the calculation load of the image processing unit 220 can be reduced, the processing speed can be increased.

In addition, in each of the above-described embodiments, only the difference in pixel values between the reduced images ps is used as the image change amount e. However, the image change amount e may be a motion vector indicating the motion of the subject between the reduced images ps.

In addition, in each of the above-described embodiments, after all the images are acquired by continuous shooting, the reduction process of reducing the frame image p to generate the reduced image ps, and the image change amount calculation of calculating the amount of change between the reduced images ps as the image change amount e are performed. deal with. However, these processes may also be performed while performing continuous shooting.

In addition, in the first embodiment, the function value a of the isolated rectangular function r[x] is the average value of each value of the image change amount array E in the motion interval D, but the function value a may be the image change amount in the motion interval D The median value of each value of E is arranged. In addition, in the first embodiment, the function b of the isolated rectangular function r[x] is the average value of each value of the image change amount array E in the non-moving interval D', but the function value b may be the non-moving interval D' The amount of image change in ranks the median value of each value of E.

In addition, in the first embodiment, the sum of the difference values between the square isolated rectangular function r[x] and the image variation array E is taken as the degree of deviation J. However, the sum of the absolute values of the differences between the isolated rectangular function r[x] and the image variation arrangement E can also be used as the degree of deviation J, and the difference between the cubic isolated rectangular function r[x] and the image variation arrangement E can also be used The sum of the values is taken as the degree of deviation J. Mainly, the degree of deviation J may be any one as long as it is the absolute difference between the isolated rectangular function r[x] and the image change amount arrangement E.

In each of the above-described embodiments, reduction processing for reducing the frame image p to generate the reduced image ps is executed. However, the amount of image change may be calculated from the frame image before reduction without performing the reduction process. However, in this case, since it is easily affected by disturbances such as hand shake, it is often necessary to perform position matching processing in hand-held shooting. Thus, the method of performing the reduction process has a smaller calculation load than the method of performing the position matching process. In addition, in the reduction process, the aspect ratio of the reduced image may be appropriately changed from the aspect ratio of the frame image before reduction.

It is also possible to combine the first embodiment and the second embodiment. For example, the motion interval D is extracted by the image extraction process of the first embodiment, and then M frames are extracted from the motion interval D by the image extraction process of the second embodiment.

The present invention is not limited to a digital camera, but can also be applied to a personal computer having a camera function and the like.

Claims (3)

1. image processing apparatus comprises:

The image input part is imported a plurality of images continuous on time sequencing;

The variable quantity operational part is calculated in a plurality of images of described input the variable quantity between image adjacent on the time sequencing respectively;

Interval specifying part, in time sequencing, specify the 1st interval successively when changing the 1st interval of in all intervals that described a plurality of images all exist, containing successively, and, specify i.e. the 2nd interval, interval behind described the 1st interval of from described all intervals, removing appointment successively successively;

The value determination section, in each image that from the 1st interval of described appointment, exists, decide the 1st value according to each the 1st variable quantity of calculating by described variable quantity operational part, and, in each image that from the 2nd interval of described appointment, exists, decide the 2nd value according to each the 2nd variable quantity of calculating by described variable quantity operational part;

The irrelevance operational part, each specify the described the 1st when interval, calculate the 1st irrelevance between described each the 1st variable quantity and described the 1st value respectively, and, each specify the described the 2nd when interval, calculate the 2nd irrelevance between described each the 2nd variable quantity and described the 2nd value respectively; With

The image extraction unit is extracted in the summation of described each the 1st irrelevance of calculating and each the 2nd irrelevance to hour the image that exists in the 1st interval by described interval specifying part appointment out.

2. image processing apparatus according to claim 1 is characterized in that:

The mean value decision of the 1st variable quantity of calculating in the image that described value determination section will be existed from described the 1st interval by described variable quantity operational part is described the 1st value, and the mean value decision of the 2nd variable quantity of calculating in the image that will be existed from described the 2nd interval by described variable quantity operational part is described the 2nd value.

3. image processing method, it comprises:

The image input step is gone up continuous a plurality of images input time in proper order;

The variable quantity calculation step is calculated in a plurality of images of described input the variable quantity between image adjacent on the time sequencing respectively;

Interval given step, in time sequencing, specify the 1st interval successively when changing the 1st interval of in all intervals that described a plurality of images all exist, containing successively, and, specify i.e. the 2nd interval, interval behind described the 1st interval of from described all intervals, removing appointment successively successively;

The value deciding step, in each image that from the 1st interval of described appointment, exists, decide the 1st value according to each the 1st variable quantity of calculating by described variable quantity calculation step, and, in each image that from the 2nd interval of described appointment, exists, decide the 2nd value according to each the 2nd variable quantity of calculating by described variable quantity calculation step;

The irrelevance calculation step, each specify the described the 1st when interval, calculate the 1st irrelevance between described each the 1st variable quantity and described the 1st value respectively, simultaneously, each specify the described the 2nd when interval, calculate the 2nd irrelevance between described each the 2nd variable quantity and described the 2nd value respectively; With

Image is extracted step out, extracts in the summation of described each the 1st irrelevance of calculating and each the 2nd irrelevance to hour the image that exists in the 1st interval by described interval given step appointment out.

Images