JP6442102B1 - Information processing system and information processing apparatus - Google Patents

Abstract

An information processing system and an information processing apparatus capable of efficiently performing work related to image production.
In an information processing system, when a sound effect is added to a moving image, a voice recognition unit recognizes a voice uttered by an operator during reproduction of the moving image data, and sound data corresponding to the recognized voice. The search unit searches for (sound effect) and acquires sound data to be added. By adding the acquired sound data to the moving image data being reproduced, reproduction data in which the sound data and the moving image data are associated with each other can be obtained. In this way, sound effects can be added to (or associated with) moving images by voice recognition, so that sound effects can be added while viewing the moving image being played back. It is possible to improve.
[Selection] Figure 2

Description

  The present invention relates to an information processing system and an information processing apparatus, and more particularly, to an information processing system and an information processing apparatus capable of associating reproduction data including image information with sound information.

  In recent years, the provision of entertainment and services using images, such as pachinko machines and slot machines, video games, WEB sites, etc., has become widespread. The production of such images requires a wide range of work, including the creation and design of the characters that make up the images, video editing, and sound editing that accompanies the images, and many people are involved in such work. Is. A tool for facilitating such work related to image production has been proposed (see, for example, Patent Document 1).

JP 2004-266721 A

  However, an image editing tool such as that disclosed in Patent Document 1 can improve the visual workability by making the configuration of the editing screen easy to understand. Since the editing work (editing process) itself is not efficient, further efficiency of work related to image production is desired.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a system and apparatus capable of efficiently performing work related to image production.

In order to solve the above-described problems, the present invention employs the following configuration.
That is, the information processing system of means 1 is
Re co means you play including playback data image information,
And recognition means for recognizing input during playback of the previous SL re-raw data,
Generating means for generating sound information based on the input recognized by the recognition means;
And related means for associating the sound information and the prior SL playback data generated by the generating means,
Storage means for storing a plurality of sound information,
The generation means acquires sound information corresponding to the input recognized by the recognition means from the sound information stored in the storage means, and generates sound information associated with the reproduction data. ,
Sound information stored in the storage means is managed by classification,
A specifying unit capable of specifying a classification of sound information to be generated by the generating unit among the classifications;
The generating means acquires sound information corresponding to the input recognized by the recognizing means from the sound information of the classification specified by the specifying means among the sound information stored in the storage means. Is the gist.

According to this, the sound information based on the input during the reproduction of playback data is generated, the association between the sound information the generation and playback data is performed, improve the efficiency of work related to the image production It becomes possible. In addition, since sound information corresponding to the input is acquired and generated from a plurality of pieces of sound information stored in advance, it is possible to diversify the associated sound information. Furthermore, by specifying the classification (type) of the sound information to be associated in advance, the sound information corresponding to the designated classification is generated based on the input during reproduction of the reproduction data and is associated with the reproduction data. Therefore, it is possible to improve the efficiency of selecting sound information to be associated.

Further, the information processing apparatus means 2 for solving the problems described above,
An information processing apparatus connectable to a reproduction processing apparatus capable of reproducing reproduction data including image information,
Recognizing means for recognizing input during reproduction of reproduction data by the reproduction processing device;
Generating means for generating sound information based on the input recognized by the recognition means;
Association instruction means for instructing the reproduction processing apparatus to associate the sound information generated by the generation means with the reproduction data;
Storage means for storing a plurality of sound information,
The generation means acquires sound information corresponding to the input recognized by the recognition means from the sound information stored in the storage means, and generates sound information associated with the reproduction data. ,
Sound information stored in the storage means is managed by classification,
A specifying unit capable of specifying a classification of sound information to be generated by the generating unit among the classifications;
The generating means acquires sound information corresponding to the input recognized by the recognizing means from the sound information of the classification specified by the specifying means among the sound information stored in the storage means. Is the gist.

According to this, since the sound information is generated and associated with the reproduction data based on the input during reproduction of the reproduction data in the reproduction processing apparatus, it is possible to improve the efficiency of the work related to image production . In addition, since sound information corresponding to the input is acquired and generated from a plurality of pieces of sound information stored in advance, it is possible to diversify the associated sound information. Furthermore, by specifying the classification (type) of the sound information to be associated in advance, the sound information corresponding to the designated classification is generated based on the input during reproduction of the reproduction data and is associated with the reproduction data. Therefore, it is possible to improve the efficiency of selecting sound information to be associated.

  According to the above-described present invention, it is possible to efficiently perform work related to image production.

It is a figure which shows the hardware structural example of the information processing system which concerns on the Example of this invention. It is a figure which shows the system configuration example of the information processing system which concerns on the Example of this invention. It is a figure which shows an example of the edit screen displayed by starting of an information processing system. It is a figure which shows an example of a genre designation | designated screen. It is a figure which shows the outline of the table of a sound effect database. It is a flowchart of an edit process. It is a flowchart of a search process. It is a flowchart of a sound data addition process. It is a figure which shows an example of an edit operation | work.

  Next, embodiments of the present invention will be described using examples.

[Entire system configuration]
FIG. 1 shows a hardware configuration example of an information processing system 10 according to an embodiment of the present invention, and FIG. 2 shows a system configuration example of the information processing system 10 according to the embodiment. The information processing system 10 of the present embodiment is configured to cause the PC to function as the information processing system 10 by installing the software (program) of the system on a commercially available personal computer (PC). For this reason, the hardware configuration of the information processing system 10 according to the present embodiment has the same configuration as that of a general personal computer. That is, the CPU 100 that controls the processing of the entire system (device), the ROM 101 that stores basic programs and data necessary for starting up and operating the PC, and the data are temporarily stored when the CPU 100 executes various processes. A RAM 102 used as a work memory for storing data, a frame buffer memory 103 (VRAM) for storing image data to be displayed on the display unit 106, which will be described later, and generating or compressing compressed image data by compressing the image data An image compression / decompression unit 104 that decompresses and reproduces image data, and an auxiliary storage unit 105 that includes an HDD or the like and stores at least a program, image data, sound data, and the like that constitute the information processing system 10 according to the present embodiment. And a display that displays various information, such as a liquid crystal display 106, an operation input unit 107 that is configured by a keyboard, a mouse, and the like for performing operation input, a voice input unit 108 that is configured by one or a plurality of microphones (sound collecting unit) and performs voice input, and 1 or A sound output unit 109 configured by a plurality of speakers and outputting various sounds. In addition, although it has the structure and function with which a general personal computer is provided, illustration is abbreviate | omitted about this.

  As shown in FIG. 2, the information processing system 10 of this embodiment that functions under such a hardware configuration can roughly reproduce reproduction data including moving image data (image information). And the information processing apparatus 30 capable of generating sound data (sound information) associated with the reproduction data reproduced by the reproduction processing apparatus 20. The reproduction processing device 20 includes a reproduction unit 21 that performs processing related to reproduction of reproduction data including image data of moving images (hereinafter also referred to as “moving image data”), and processing related to editing of moving image data and sound data. And an editing unit 22 that executes the processing and a reading unit 23 that executes processing relating to reading of sound data generated by the information processing device 30. The editing unit 22 is configured to be able to execute various types of editing processing such as adding the sound data read by the reading unit 23 to the moving image data and adjusting the reproduction position of the sound data with respect to the moving image data. Yes. Further, the playback unit 21 is configured to be able to execute processing related to playback of playback data including sound data and moving image data edited by the editing unit 22.

  The reproduction data reproduced by the reproduction processing device 20 includes one or a plurality of moving image data (image information), and one or a plurality of sound data (sound information). Some of them include both moving image data (image information) and sound data (sound information). The moving image (moving image data) is composed of a plurality of frame-by-frame images (image data) arranged in time series. Hereinafter, reproduction data including moving image data may be simply referred to as moving image data.

  The information processing apparatus 30 recognizes the speech recognition unit 31 from among a plurality of sound data stored in the auxiliary storage unit 105 and a speech recognition unit 31 that performs processing related to recognition of speech input by the speech input unit 108. The search unit 32 that executes processing related to the search of sound data corresponding to the sound that has been played, and the sound data acquisition unit 33 that executes processing related to acquisition of the sound data searched by the search unit 32 and supply to the playback processing device 20 It consists of. When the activation instruction for the information processing system 10 is input from the operation input unit 107, the CPU 100 loads the system program stored in the auxiliary storage unit 105 into the RAM 102 in response to the input, and executes the program. As a result, the information processing system 10 is activated, and processing by each unit shown in FIG. 2 can be executed.

[Editing screen]
Next, an editing screen displayed on the display unit 106 when the information processing system 10 is activated will be described. FIG. 3 shows an example of an edit screen according to the information processing system 10 of the present embodiment. As shown in FIG. 3, the editing screen 200 is composed of a plurality of display areas (windows). Specifically, a moving image title display area 201, a reproduction display area 202, a timeline display area 203, A sound editing display area 204 and a voice recognition display area 205 are included. Note that the configuration of the editing screen 200 described in this embodiment is merely an example, and various types of display areas (windows) constituting the editing screen, the number, size, arrangement, display contents, and the like may be employed. Is possible.

  The moving image title display area 201 is an area for displaying titles (file names) of moving image data (moving image files) stored in a predetermined data folder (storage area) of the auxiliary storage unit 105, and lists a plurality of titles. It is configured so that it can be displayed. A person who performs work related to image editing (image production) using the information processing system 10 (hereinafter also referred to as “worker”) operates a mouse or the like as the operation input unit 107 to display a moving image title display area. By clicking the title of the moving image data to be edited from the titles displayed in 201, the moving image data to be edited can be selected. When the moving image data to be edited is selected, the image at the reproduction start position of the first frame of the moving image data is displayed in a stationary state in the reproduction display area 202.

  The reproduction display area 202 is an area for reproducing and displaying a moving image based on the moving image data to be edited selected in the moving image title display area 201. Media operation icons such as “play”, “stop”, and “pause” are provided at the bottom of the playback display area 202. The operator operates the mouse or the like as the operation input unit 107 to operate the media. By clicking the icon, it is possible to input an instruction such as playback or pause of the moving image displayed in the playback display area 202. As described above, when the “playback” icon is clicked while the moving image data to be edited is selected in the moving image title display area 201, reproduction of the selected moving image data (playback data) to be edited is started. A moving image related to the data is reproduced and displayed in the reproduction display area 202. In addition, on the left side of the media operation icon in the playback display area 202, a playback time display unit for displaying in real time the progress of the playback time of the moving image being played back is provided. Furthermore, a “Preview” button is provided on the right side of the media operation icon. When the “Preview” button is pressed (clicked), the reproduction data being edited (moving image data to be edited and sound data are combined). Playback data) is played back from the beginning.

  The timeline display area 203 is an area for displaying moving images reproduced and displayed in the reproduction display region 202, that is, reproduction position information (moving image timeline) on the time axis of moving image data to be edited. In the timeline display area 203, one timeline cursor TC extending in the vertical direction across the sound editing display area 204 is displayed. The timeline cursor TC indicates the current playback position, and moves in the time axis direction (from the left side to the right side in FIG. 3) as the playback time elapses. The timeline display area 203 also displays the waveform HK of the sound data when the reproduction data of the moving image reproduced and displayed in the reproduction display area 202 includes sound data such as BGM (background music). It is configured. That is, the timeline display area 203 can also display reproduction position information (sound timeline) on the time axis of sound data. As a result, the operator looks at the timeline display area 203, so that the correspondence between the reproduction position of the moving image reproduced and displayed in the image reproduction display area 202, the sound reproduced accompanying the moving image, and the reproduction position are displayed. It is possible to easily grasp information useful for editing work, such as the quality and strength of the output sound.

  The sound editing display area 204 is an area for displaying information related to sound editing when sound data (sound information) is added (added) to moving image data (reproduction data, first reproduction data) to be edited. In the information processing system 10, when a voice uttered by an operator or the like is recognized through a microphone or the like as the voice input unit 108 while a moving image is played back and displayed in the playback display area 202, the recognized voice is supported. The sound effect to be added can be added to the playback position on the timeline (on the time axis) in accordance with the voice recognition timing (substantially synchronized). Note that the voice recognition timing here is substantially the same as the utterance timing that triggered the voice recognition. Corresponding to this, in the sound editing display area 204, information of sound data (effect type (title), time indicating playback timing, etc.) related to the sound effect added based on the speech recognition is displayed. Displayed according to the playback position.

  Here, in the present embodiment, when a sound effect (sound data) is added, a mark MK indicating the position where the sound effect is added is displayed on the time axis (on the time line) of the timeline display area 203. In the sound editing display area 204, a sound icon IC indicating information of the added sound data is displayed (arranged) in association with the mark MK. Thus, the sound effect is added to the moving image (moving image data to be edited) reproduced and displayed in the reproduction display region 202 by the operator looking at the timeline display region 203 and the sound editing display region 204. It is possible to easily grasp the contents of the added sound effect, the correspondence between the moving image being reproduced and displayed, and the reproduction position of the sound effect, and the like. In the present embodiment, as shown in FIG. 3, the mark MK and the sound icon IC are connected by a broken line to associate them. FIG. 3 shows an example in which three marks MK and three sound icons IC are displayed (marks MK1 to MK3, sound icons IC1 to IC3).

  Further, in this embodiment, the operator operates the mouse or the like as the operation input unit 107 and drags the sound icon IC displayed (arranged) in the sound editing display area 204 in the left-right direction. The playback position of the sound effect (sound data) corresponding to the icon IC can be adjusted. At this time, the mark MK (and the broken line) displayed on the time axis of the timeline display area 203 is also configured to move in the left-right direction in conjunction with it, so that fine adjustment of the sound effect playback position is facilitated. Can be done. Further, in a state where one or more sound icons IC are displayed in the sound editing display area 204, that is, in a state where sound data of one or more sound effects is added to the moving image data (during editing), the reproduction display area When the “Preview” button 202 is pressed (clicked), moving image data to which sound data corresponding to the sound icon displayed in the sound editing display area 204 is added, that is, the sound data and moving image data to be edited are displayed. The reproduction data (second reproduction data) obtained by synthesizing the data is reproduced from the beginning. As a result, the moving image is reproduced and displayed in the reproduction display area 202 from the reproduction start position of the first frame, and a sound effect is output from the sound output unit 109 (speaker) when the reproduction position (reproduction timing) of the added sound data is reached. Is done.

  In this embodiment, when sound effect sound data is added on the timeline (time axis) of the moving image data, that is, when the sound icon IC is displayed (arranged) in the sound editing display area 204. Accordingly, a sound effect based on the sound data is output from the sound output unit 109. Thereby, the operator can sense the impression of the moving image to which the sound effect is added in real time. Furthermore, by clicking the sound icon IC displayed in the sound editing display area 204 and inputting a deletion instruction, the sound data corresponding to the sound icon IC (that is, the added sound effect) is deleted ( It is configured so that the association with the reproduction data can be released.

  The voice recognition display area 205 is an area for displaying the contents of the voice recognized by the voice recognition unit 31 via a microphone or the like as the voice input unit 108. The voice recognition display area 205 is configured to display voice (recognized voice) uttered by an operator or the like as text. In this embodiment, when reproduction of moving image data to be edited selected in the moving image title display area 201 is started, sound effects (sound data) can be added by voice recognition in conjunction with this. (Hereinafter also referred to as “voice recognition mode”). For this reason, in this embodiment, during the speech recognition mode, the speech recognition display area 205 is configured to display the content (text) of the recognized speech in chronological order from top to bottom. By displaying the content (text) of the voice recognized in this way, it is possible to confirm whether the content of the utterance (spoken voice) is correctly recognized. Note that the voice recognition mode is configured to end when the reproduction of moving image data is stopped.

  Further, a “genre designation” button is provided at the lower right of the sound editing display area 204. When the “genre designation” button is pressed (clicked), the genre designation screen 210 shown in FIG. It is configured to be displayed superimposed on the side. In this embodiment, as will be described later, sound effects (sound data) that can be added to a moving image are managed for each genre (by classification). The genre designation screen 210 for designating the genre (classification) of (sound data) is configured to be displayed. The genre designation screen 210 of the present embodiment is configured by forming a check box for each genre. By operating a mouse or the like as the operation input unit 107 and checking a check box of a genre to be designated, It is possible to select and specify one or more genres of sound effects to be added. Genres that can be designated (selectable) on the genre designation screen 210 correspond to genres registered in a sound effect database to be described later (see FIG. 5). When the genre of the sound effect is designated, the sound effect to be added is searched for the sound effect of the designated genre (S108 described later). Further, the genre designation screen 210 is configured to finish the display on the display unit 106 by inputting a predetermined display end instruction through the operation input unit 107.

[Edit processing]
Next, operation processing of the information processing system 10 of the present embodiment will be described with reference to FIG. The operation processing of the system 10 is executed by cooperation between the reproduction processing device 20 and the information processing device 30. When the system 10 is activated, the CPU 100 displays the above-described editing screen 200 (see FIG. 3) on the display unit 106 and executes the editing process shown in FIG.

  In S100, it is determined whether or not reproduction of moving image data is being stopped (including pause). As a result, when it is determined that it is not stopped (NO in S100), that is, when moving image data is being reproduced, the process proceeds to S104 described later, and it is determined that it is stopped (YES in S100). Then, it is determined whether or not an input for instructing the start of reproduction of moving image data has been performed (S101). The instruction to start playback is input by clicking the “play” icon among the media operation icons displayed in the playback display area 202 of the editing screen 200. If it is determined that there is no playback start instruction input (NO in S101), the process proceeds to S110 described later, and if it is determined that there is a playback start instruction input (YES in S101), the playback unit 21 selects a moving image title. The reproduction of the moving image data to be edited selected in the display area 201 is started (S102), and the voice recognition unit 31 sets the voice recognition mode (S103). As a result, a moving image based on the moving image data to be edited is reproduced and displayed in the reproduction display area 202.

  Next, in S104, it is determined whether or not voice input has been performed by the voice input unit 108. If it is determined that there is no voice input (NO in S104), the process proceeds to S110 described later, and there is a voice input. If it is determined (YES in S104), the voice recognition unit 31 acquires the input voice and performs voice recognition (S105). The voice recognition unit 31 is a voice recognition function capable of recognizing the voice of an operator or the like input via the voice input unit 108, and performs voice recognition using a general voice recognition algorithm. As the voice recognition unit 31, a known voice recognition function can be used. If the input voice cannot be properly recognized (NO in S106), the process proceeds to S110 described later, and if it can be recognized (YES in S106), The recognized voice is converted into text data, and the content of the voice based on the text data is displayed in the voice recognition display area 205 of the editing screen 200 (S107), and the search unit 32 performs a search process based on the text data. Is executed (S108).

  Here, sound data (sound information) that can be added (associated) with moving image data (reproduction data, first reproduction data) and sound data are actually added (associated) to the auxiliary storage unit 105. A sound effect database to be referred to is stored. The sound data stored in the auxiliary storage unit 105 is obtained by converting sound that will be used as a sound effect to be added to a moving image, and in this embodiment, a natural phenomenon, the movement of an object, and the sound produced by the object. Sound data created from sound of various genres, such as animal calls, is used as sound data. The contents of the sound effects related to the sound data are expressed in language (onomatopoeia) as speech (hereinafter also referred to as “recognition words”) to be recognized when the sound data is added by speech recognition. A sound effect database defining the relationship between the recognition word and sound data (sound effects) is stored in the auxiliary storage unit 105.

  FIG. 5 shows an outline of the table structure of the sound effect database of the present embodiment. As shown in FIG. 5, the table of the sound effect database of the present embodiment is a pseudo-spoken word (““ natural phenomenon ”etc.) prepared by verbalizing and expressing the content of each sound effect for each genre of sound effect prepared in advance (“ And the like (ie, data storage destination, file name, etc.) corresponding to the sound effect data corresponding to this. The information processing system 10 according to the present embodiment updates (adds, deletes, etc.) sound data (sound effects) stored in the auxiliary storage unit 105 and updates (adds, deletes, etc.) the sound effect database corresponding thereto. It is configured to be able to do things afterwards (so-called version upgrade). In FIG. 5, for convenience of explanation, three genres of sound effects and three recognition words and sound data of each genre are illustrated, but actually, more genres, recognition words and sound data are provided. The sound data (sound effects) that can be added is diversified. In addition, FIG. 5 illustrates an example in which the sound data of each genre is not included in other genres (does not overlap), but sound data included in (overlapping) multiple genres may exist. .

  In this embodiment, sound data (sound effects) is managed using a sound effect database having a table structure as shown in FIG. 5, and the search unit 32 refers to the sound effect database to search for sound data. (S108) is executed. This search process (S108) is executed according to the flowchart shown in FIG. That is, in the search process (S108), first, a command sentence for searching and acquiring sound data corresponding to the voice (recognition word) recognized in S106 is generated (S201). According to the generated command sentence, the sound effect database is accessed (S202), the sound data corresponding to the recognized voice (recognition word) is searched (S203), and the sound data is specified as a result of the search. The identified sound data is acquired from the sound data stored in the auxiliary storage unit 105 (S204). As a result, sound data of sound effects to be added to the moving image data (reproduction data, first reproduction data) currently being reproduced is generated (extracted).

  For example, when the voice uttered by the worker using the system 10 is “don't care”, when the voice recognition unit 31 recognizes this voice (YES in S106), the search unit 32 stores the sound effect database. Referring to the sound data of the sound effect corresponding to the recognized voice “Dokan” (recognition word) is searched (S203). As shown in FIG. 5, the sound data of the sound effect corresponding to the recognition word “Dokan” is “Sound data C2 (explosive pronunciation)”, and therefore, among the sound data stored in the auxiliary storage unit 105 "Sound data C2" is acquired from (S204).

  Here, when the voice (recognition word) uttered by the operator overlaps with a plurality of genres, for example, when the recognition word is “about” shown in FIG. 5, the search unit 32 makes a sound corresponding to “about”. Data A1 (natural phenomenon: lightning sound) and sound data B1 (object movement: rolling sound) are extracted as search results. In this embodiment, there are a plurality of sound data corresponding to one recognition word in this way. In this case, one sound data is randomly specified (extracted) from the plurality of sound data and acquired. Note that the method of specifying (extracting) one sound data when there are a plurality of sound data corresponding to one recognition word is not limited to random extraction, and the conditions for specifying one sound data are determined in advance in the program. It is possible to specify based on the conditions. For example, a priority order is set for each sound data with a common recognition word, and one sound data is specified according to the priority order, or one sound data corresponding to the recognition word is specified according to the characteristics of the recognized speech. It is possible to identify the type (title, genre, etc.) of moving image data to be edited (playing back) and specify one sound data suitable for this. Also, by specifying the genre of the sound effect in advance on the genre specification screen 210 described above, it is possible to avoid duplication of recognition words (sound data), and the efficiency of sound data search (selection) can be reduced. It is possible to improve.

  In this embodiment, when the result of the search by S203 is 0 and the sound data cannot be specified (when the recognition word is not supported), a message image indicating that fact is displayed on the editing screen 200, The process proceeds to S110 without performing S109. In this case, sound data of sound effects is not generated, and accordingly, sound data (sound effects) is not added to the moving image data.

  Returning to FIG. 6, when the CPU 100 finishes the search process (S108), the sound data acquired (generated) in S204 of the search process is converted into moving image data (playback data, first playback data) currently being played back. A sound data adding process (S109) for adding is executed. This sound data addition process (S109) is executed according to the flowchart shown in FIG. That is, in the sound data addition process (S109), first, the instruction unit 33 reads the sound data acquired (generated) in the previous S204, and the sound data and the moving image data being reproduced, to the reproduction processing apparatus 20. (S301) is output (S301). When the reproduction processing device 20 receives the instruction signal from the instruction unit 33 (information processing device 30), the reading unit 23 reads the sound data acquired (generated) in the previous S204 (S302), and the read sound data and Then, the editing unit 22 synthesizes the moving image data (reproduction data, first reproduction data) currently being reproduced (S303). In S303, a process of pasting the sound data read in S302 in association with the current reproduction position information (reproduction time information) in the moving image data (reproduction data) currently reproduced is pasted to the reproduction data (first reproduction data). Executed. Thereby, the sound data of the sound effect corresponding to the voice uttered by the operator is substantially synchronized with the utterance timing (voice recognition timing) with respect to the moving image data (reproduction data, first reproduction data) currently being reproduced. Will be added (associated). At this time, in the timeline display area 203 and the sound edit display area 204 of the edit screen 200, a mark MK and a sound icon IC are displayed at the reproduction positions where the current sound data is to be added (FIG. 3). Further, in this embodiment, when the sound data and the moving image data are synthesized in S303, the sound data is reproduced by the reproduction unit 21, and a sound effect based on the sound data is output from the sound output unit 109. .

  Returning to FIG. 6, when the sound data adding process (S109) is completed, the CPU 100 determines whether or not an instruction to end the editing process has been input (S110). The input of the end instruction is performed by, for example, operating the mouse or the like as the operation input unit 107 and clicking the end icon (x mark) on the upper right of the editing screen 200. If it is determined in S110 that an end instruction has been input (YES in S110), the editing process is terminated, and the operation of the information processing system 10 is terminated. On the other hand, when it is determined that the end instruction is not input (NO in S110), the process returns to S100, and the processes of S100 to S110 described above are repeated. During this time, it is possible to continue adding sound data until the reproduction of the moving image data to be edited is completed. Therefore, it is possible to add one or a plurality of sound data (sound effects) to one moving image data. Information relating to the moving image data and sound data synthesized in S303, that is, information such as the type of moving image data being reproduced, the type of sound data to be synthesized with the moving image data, and the composition position (reproduction position) is edited. While processing is being performed (during editing), the data is stored in a predetermined storage area such as the RAM 102. Further, when the editing work on the moving image data to be edited is finished, an instruction input for confirming the result of the work is performed, so that the edited moving image data (moving image data to which sound data is added, second reproduction is performed). Data) is stored in a predetermined storage area (folder or the like) of the auxiliary storage unit 105.

  The editing process described above is the main operation process of the information processing system 10 of the present embodiment. In addition to this, the CPU 100 is configured to be able to execute a process related to editing work by the system 10. For example, while the system 10 is activated and editing is performed, one or more sound data is added to the moving image data to be edited, that is, the sound icon IC is displayed in the sound editing display area 204 as shown in FIG. When the “preview” button provided in the reproduction display area 202 is pressed in a state where the sound icons IC1 to IC3 in FIG. 3 are displayed, the editing unit 22 then displays the reproduction display area 202 at that time. The image data (moving image data to be edited) related to the moving image displayed on the screen and the sound data (sound data of the added sound effect) related to the sound icon IC displayed in the sound editing display area 204 are combined. Then, the edited reproduction data (second reproduction data) is created, and the reproduction unit 21 reproduces the reproduction data. Then, a moving image based on the reproduction data is reproduced and displayed in the reproduction display area 202, and a sound effect based on the reproduction data is output from the sound output unit 109 (speaker). Thus, the operator who performs the editing work can reproduce the moving image with the sound effect added from the beginning and check the editing content.

  Further, for example, while the system 10 is activated and editing is performed, the sound icon IC displayed in the sound editing display area 204 is moved (dragged) in the left-right direction (time axis direction of the timeline) by a mouse operation. Then, the editing unit 22 changes and edits the reproduction position information (reproduction time information) of the sound data corresponding to the sound icon IC according to the movement of the sound data on the timeline (on the time axis). Update the association with the target moving image data. Thereby, the playback position of the sound data (sound effect) added to the moving image data is changed, and the playback position of the sound data (sound effect) can be adjusted afterwards.

  Note that the setting of the voice recognition mode is canceled when executing reproduction data creation / reproduction processing based on pressing of the “preview” button or sound data reproduction position (reproduction timing) change processing by moving the sound icon IC, The voice recognition unit 31 (voice recognition function) is configured not to work.

[Example of editing work]
Next, an example of work (editing work) related to image production using the information processing system 10 according to the present embodiment will be described. Here, as shown in FIG. 9, the editing screen 200 (see FIG. 3) is displayed on a large-sized liquid crystal monitor connected to a notebook personal computer (notebook PC) functioning as the system 10. A description will be given assuming that there are five workers (workers A to E) who are involved in the work. Further, it is assumed that one omnidirectional (omnidirectional) microphone is used as a voice input microphone (voice input unit 108). The microphone as the voice input unit 108 can be a unidirectional microphone. In this case, microphones for the number of workers are prepared, or one microphone is used by all workers. Or the like. Further, the microphone as the voice input unit 108 may be wired or wireless. Furthermore, it is assumed that various sounds are output from a speaker (sound output unit 109) connected to the notebook PC by wire or wirelessly.

  In starting the work, first, the present system 10 is activated and the editing screen 200 is displayed on the monitor (display unit). Then, moving image data (reproduction data) to be edited is selected in the moving image title display area 201 on the editing screen 200, and a moving image based on the data is reproduced and displayed in the reproduction display area 202. While the reproduction display is progressing, each worker utters an onomatopoeia (recognition word) of the sound effect at a timing when he / she wants to add the sound effect while watching the moving image being reproduced and displayed. When the spoken voice (recognition word) is input to the information processing apparatus 30 via the microphone (voice input unit 108) and recognized by the voice recognition unit 31, sound data of a sound effect corresponding to the recognized voice is obtained. It is generated and added to the moving image data (reproduction data, first reproduction data) to be edited. At this time, a sound icon IC indicating information regarding the added sound effect (sound data) is displayed in the sound edit display area 204 of the edit screen 200, and the sound effect is output from the speaker (sound output unit 109). Be done

  In the present system 10, sound effects (sound data) due to such utterances can be freely added by each of the five workers A to E, and sound data (sound effects) based on the voice recognition of each person. ) Can be added. When it is desired to confirm the content of the moving image to which the sound effect is added (reproduced data after editing), the effect is obtained by pressing (clicking) the “preview” button (see FIG. 3) in the reproduction display area 202. A moving image with sound (reproduction data obtained by synthesizing sound data and moving image data, second reproduction data) is reproduced and can be immediately confirmed.

  As described above, the editing operation can be efficiently performed by proceeding the operation while adding the sound effect (sound data) by the voice recognition by several persons (here, five persons). Specifically, for example, bring sample data of a moving image with sound effects created by a person in charge of voice design to another worker, work leader, customer, etc., and sound effects based on the sample data When the attached moving image is confirmed, if the operating environment of the system 10 is in place, it is possible to correct, adjust, and change the sound effect on the spot. Moreover, the recognition word when adding sound effects by voice recognition is made to be onomatopoeia, and it is possible to generate and add sound effects corresponding to this, so that the image of each person on the spot has an image It is possible to add sound effects that are as close as possible. Therefore, as a result of confirming the moving image with sound effects based on the sample data, even if it is necessary to correct the sound effects, the data in the form based on the intention of another person in charge is immediately Since it can be created, it is not necessary for the sound person to take home the sample data and make corrections. Thereby, the efficiency of the editing work is greatly improved.

  In the information processing system 10 according to the present embodiment described above, when sound data of sound effects is added (associated) to reproduction data (first reproduction data) including moving image data, the reproduction data is being reproduced ( It is possible to recognize a voice (recognition word) uttered by an operator during playback and display of a moving image and add (associate) sound data (sound effect) corresponding to the recognized voice. In this way, by adding sound effects to a moving image so as to be able to be performed by voice recognition during the reproduction and display of the moving image, the sound effect can be obtained simply by speaking while confirming the reproduced and displayed moving image. Can be added at an arbitrary timing, so that it is possible to improve the efficiency of work relating to image production.

  In particular, in this embodiment, the recognized speech (recognition word) is an onomatopoeia, and sound data of sound effects corresponding to the recognition word is managed by the sound effect database. And since it is comprised so that sound data (sound effect) may be added to moving image data (reproduction data, 1st reproduction data) based on the onomatopoeia (recognition word) which the operator uttered, the image which a worker has This makes it possible to proceed editing work sensuously. Further, as in the example of the editing work described above, a plurality of people can be involved in the work of adding sound data (sound effect) to the same editing target moving image data (playback data, first playback data). It becomes. As a result, it is possible to improve work efficiency related to image production.

  It should be noted that the present invention can adopt a configuration (hereinafter also referred to as “modification”) different from the above-described embodiments. Hereinafter, modified examples will be described.

[Modification 1]
In the information processing system 10 of the above-described embodiment, there is no particular limitation on the voice that can be recognized by the voice recognition unit 31 via the voice input unit 108. However, a limitation can be provided on this. For example, a voice authentication function may be added to the above-described embodiment, and the voice recognition unit 31 may recognize only the voice of a person who has been registered as a system user in advance. As the voice authentication function, a known function can be used. As a system configuration according to the first modification, for example, a registration unit that registers information (speech information) related to a voice of a person who performs editing work using the system 10, and a voice that is input via the voice input unit 108. The information processing device 30 is provided with an identification unit for identifying whether or not the voice of the person registered in the registration unit is present, and the voice recognition unit 31 recognizes only the voice of the person corresponding to the information registered in the registration unit. The configuration can be exemplified. In such a configuration, first, a person using the system 10 registers his / her voice via the voice input unit 108 in advance. This registration is performed by the input unit analyzing the input voice, extracting characteristic data (frequency, etc.) of the voice and recording it as a voice model (voice information) of a specific individual. When the editing work is actually performed using the system 10, the worker inputs his / her predetermined voice through the voice input unit 108 when the use of the system 10 is started. The identification unit compares and compares the input voice with the voice model (voice information) registered (recorded) in the registration unit. If the two match, the worker who has input the voice It becomes possible to add sound data by the above-described speech recognition. According to the first modified example, only a person who has registered in advance can add sound data by voice recognition. Therefore, sound data is added by recognizing the voice of a person who does not perform editing work. There is nothing. In addition, since it is possible to limit the number of workers who use the system at one time, it is easy to maintain an environment in which voice recognition (voice recognition function) functions properly.

[Modification 2]
In the information processing system 10 of the embodiment described above, the voice recognition unit recognizes the voice input via the voice input unit 108, and adds sound data corresponding to the recognized voice (recognition word) to the moving image data. It was configured as follows. On the other hand, the sound data may be added to the moving image data based on an input other than the voice input. For example, the operation input unit 107 provided in a personal computer (PC) such as a keyboard and a mouse, and the effect Selection (search) of sound data to be added to moving image data and sound based on input (operation input) by an operator using a dedicated controller, input screen, etc. for inputting a recognition word registered in the sound database You may comprise so that addition of data may be performed. In this case, an input recognition unit capable of recognizing input from the operation input unit 107, the controller, the input screen, or the like is provided in the information processing apparatus 30, and sound data (sound effect) corresponding to the input recognized by the input recognition unit is acquired. It can be configured to be added to moving image data. In addition, a sound generation unit capable of generating sound data such as an electronic sound based on an input from the operation input unit 107 or a controller is provided in the information processing apparatus 30, and the sound data generated by the sound generation unit is added to the moving image data. You may comprise as follows. In these configurations as well, as in the above-described embodiment, any video can be reproduced while being displayed in the reproduction display area 202 of the editing screen 200, the timeline cursor TC displayed in the timeline display area 203, etc. Any sound effect (sound data) can be added by operating the operation input unit 107, the controller, or the like at this timing. According to the second modified example, since the sound data is not added by the recognition of the voice of the person who does not perform the editing work and other surrounding noises, it is necessary to consider the noise concerned in the case of the voice recognition. Disappear.

[Modification 3]
In the information processing system 10 of the embodiment described above, sound data (sound effects) is managed by a sound effect database having a table structure as shown in FIG. 5, and the sound (recognition word) recognized by the speech recognition unit is used. Corresponding sound data is acquired and added to moving image data. When the voice recognized by the voice recognition unit does not exist as a recognition word in the sound effect database (when the recognition word is not supported), no sound data is generated and no sound data is added to the moving image data. It was. On the other hand, when there is no recognition word corresponding to the speech recognized by the speech recognition unit, one sound data approximating the recognized speech is stored as one or more sounds stored in the auxiliary storage unit 105. The data may be generated by being combined and added to (associated with) the moving image data. For example, as shown in FIG. 5, the sound data C1 of the sound effect corresponding to the recognition word “Kan” exists, but the voice recognized by the voice recognition unit is “Kankan”. If there is no recognition word and sound data to be recognized, two pieces of sound data C1 of “Kan” can be synthesized (combined) to generate one sound data (“Kankan”). In addition to sound effect sound data (sound effect data) corresponding to the sound effect database as in the embodiment, sound data corresponding to a single sound such as direct sound, stuttering sound, clear sound, muddy sound, semi-turbid sound, nasal muddy sound (monophonic data) ) Is stored in the auxiliary storage unit 105, and the single sound data and the sound effect data are synthesized to generate a single sound data (for example, “pa” + “ka” = “pa ka”)) It is also possible to synthesize data and generate one sound data (for example, “Kyu” + “I” + “N” = “Kyuin”). According to Modification 3, it is possible to eliminate as much as possible that sound data is not generated and sound effects are not added to the moving image depending on the speech recognized by the speech recognition unit. Also, it is possible to diversify the sound effects that can be added by voice recognition.

[Modification 4]
In the information processing system 10 of the above-described embodiment, when the sound data is added to the moving image data being reproduced by voice recognition, the current reproduction position information (reproduction time information) of the moving image data being reproduced. ), The sound data is added in accordance with (substantially in synchronization with) the utterance timing (voice recognition timing) of the speaker. On the other hand, the position (sound reproduction timing) to which the sound data is added to the moving image data can be made slightly earlier (advanced) than the actual speech timing (speech recognition timing). Also good. For example, when a moving image to be edited reproduced and displayed in the reproduction display area 202 is first seen by the worker, it is assumed that the utterance timing of the worker is delayed. In view of this, a sound setting screen (not shown) is provided in the same manner as the genre specification screen 210 of the above-described embodiment, and a setting (sound reproduction timing) for adding a sound effect is set slightly earlier through the screen. Configure so that it can be done in advance (additional position setting function). According to Modification 4 as described above, a user-friendly system can be achieved.

[Modification 5]
In the information processing system 10 of the above-described embodiment, the sound data and the moving image data being reproduced are synthesized in the sound data addition processing (S109) for adding the sound data acquired by the search processing (S108) to the moving image data. (S303) The reproduction unit 21 immediately reproduces the synthesized sound data, so that the sound effect based on the sound data is output by the sound output unit 109 along with the addition to the moving image. On the other hand, in the sound data addition process (S109), the reproduction position information (reproduction time information) of the sound data associated with the moving image data being reproduced is stored without synthesizing the sound data and the moving image data. The sound data may not be reproduced. That is, in the sound data addition process (S109), the sound data itself is not added, and information related to the sound data (sound data type, title, etc.) is displayed as the current reproduction position information (reproduction of the moving image data being reproduced). The time information is stored in association with the time information. In this case, although the sound data search (S203) by the search process (S108) is performed, the sound data acquisition (S204) based on the search result acquires information about the sound data (sound data type, title, etc.). This information is read in the sound data addition process (S109) and stored in association with the current reproduction position information (reproduction time information) of the moving image data (S302). Then, based on an input for instructing reproduction of the synthesized sound data and moving image data to be edited, for example, when the “Preview” button in the reproduction display area 202 is pressed, the editing unit 22 The sound data corresponding to the information stored in S302 is read from the auxiliary storage unit 105, the sound data and the moving image data to be edited are combined, and the combined reproduction data is reproduced. According to the fifth modification, the processing load on the CPU during reproduction of the moving image data being edited is reduced, so that the editing operation can be made smoother. Note that in the fifth modification as well, regarding the screen display during editing work, the editing screen 200 as shown in FIG. 3 can be displayed as in the above-described embodiment. In this way, in the sound data addition process (S109), internally, the sound data and the moving image data are not synthesized, but the reproduction position information (reproduction time) of the sound data associated with the moving image data (reproduction data) is used. Even if it is stopped to store (information), it will be easy for the operator to easily understand the state of addition of sound data to moving image data (reproduction data).

[Modification 6]
In the information processing system 10 of the above-described embodiment, the voice (recognition word) recognized by the voice recognition unit is used as an onomatopoeia, and sound data (sound effect) is added to the moving image data based on the onomatopoeia. The target of speech recognition was premised on Japanese (see FIG. 5). On the other hand, with respect to the language targeted for speech recognition, a language other than Japanese may be targeted. For example, regarding an onomatopoeia representing “dog cry”, there are many onomatopoeia having the same content, such as “Wanwan” in Japanese and “Bowwow” in English, but having different expressions (recognition words) depending on the language. Corresponding to this, the recognition word in the sound effect database is set not only for Japanese but also for other languages such as English, so that a search according to the language is possible. A language designation screen (not shown) is provided in the same manner as the genre designation screen 210 of the above-described embodiment, and is configured so that it is possible to designate in advance in which language voice recognition is to be performed through the screen (language designation). function). According to the sixth modified example, it is possible to deal with not only Japanese but also languages of other countries, so that a highly convenient system can be achieved.

[Modification 7]
In the information processing system 10 of the above-described embodiment, the sound effect database and sound data are stored in the auxiliary storage unit 105 provided in the personal computer (PC) constituting the system, and the sound effect database is accessed to perform a search process (S108). ). On the other hand, a storage unit that stores a sound effect database and sound data is provided in a server that can be connected to a PC that constitutes the information processing system 10 via a network such as the Internet, and search processing is performed on the server side. May be configured. In this case, the sound data extracted by the search process on the server side is acquired by the PC side via the network, and the acquired sound data and moving image data are synthesized. According to the seventh modified example, editing work using the present system can be performed in various places as long as the network environment is in place. In addition, the provider (seller, manufacturer, administrator, etc.) of this system can perform the operations such as updating and maintenance of the sound effect database and sound data without entrusting the system user. Therefore, it is possible to make the system more convenient to use.

  As mentioned above, although an example and a modification were explained as an embodiment of the present invention, the present invention is not limited to these, and unless it deviates from a range indicated in each claim, it limits to a statement word of each claim. However, it is possible to appropriately add improvements based on the knowledge that a person skilled in the art normally has, and to the extent that those skilled in the art can easily replace them.

  For example, in the above-described embodiments, sound data (sound effects) is associated with moving image data by voice recognition. However, for example, image data of various effect images may be associated with moving image data. In this way, it is possible to perform work related to associating one or both of sound effects and effect images with moving image data (reproduction data) by voice recognition.

  In the above-described embodiment, the personal computer (PC) is used as the software (program) of the information processing system 10 constituted by the reproduction processing device 20 and the information processing device 30 so that the PC functions as the information processing system 10. For example, when the software (program) of the information processing apparatus is installed on the PC on which the software (program) of the commercially available reproduction processing apparatus is already installed, the same as in the above-described embodiment etc. You may comprise so that an information processing system can be constructed | assembled. In this case, the information processing apparatus is configured to be connectable and communicable with the reproduction processing apparatus already provided in the PC, and configured so that both apparatuses are connected by starting the reproduction processing apparatus and the information processing apparatus. To do. In addition, by activating both devices, the display unit of the PC has a screen corresponding to the playback processing device (hereinafter also referred to as “first screen”) and a screen corresponding to the information processing device (hereinafter referred to as “second screen”). Are also displayed independently (in separate windows). Here, for example, the first screen can be composed of other areas 201 to 204 excluding the voice recognition display area 205 in the editing screen 200 (see FIG. 3) of the above-described embodiment, For example, the second screen can be composed of the voice recognition display area 205 in the editing screen 200.

  As described above, even in the configuration in which the software of the reproduction processing device and the software of the information processing device are installed separately, the moving image data (reproduction data) reproduced by the reproduction processing device is similar to the above-described embodiment. Thus, the sound data generated by the information processing apparatus can be added (associated). Further, according to such a configuration, since only the software for the information processing apparatus can be provided alone in addition to the software of the entire information processing system (reproduction processing apparatus + information processing apparatus), the reproduction processing apparatus already has For the user (user) who owns the PC on which the software is installed, it is possible to easily introduce the work environment related to image production by the system described in the embodiments and the like, which is convenient for the user (user). Rise.

  The second screen described above instructs a “start” button (not shown) for instructing the start of voice recognition (setting of the voice recognition mode) and the end of voice recognition (resetting the voice recognition mode). An “end” button (not shown) may be provided. This is because the playback processing device and the information processing device are started separately (separate software), instructions relating to the start and end of playback of moving image data (playback data) in the playback processing device, and audio in the information processing device This is because it is possible to make a separate system for the instructions related to the start / end of recognition. In this case, for example, the operator clicks the “play” icon (see FIG. 3) on the first screen and inputs an instruction to start playback of moving image data, and then “start” on the second screen. By clicking the “” button and inputting a voice recognition start instruction, the editing operation can be performed in the same manner as in the above-described embodiment.

In addition, the information processing system and information processing apparatus according to the present invention are used to produce various game effect images used in gaming machines such as pachinko gaming machines and slot machines, home games, mobile games, net games, arcade games, etc. Production of various game images used in game machines, production of various WEB site images used in advertising on the WEB site, production of various animation images used in television and movies, etc. It can be used in production. In particular, the present invention is useful in the gaming machine field, the game field, and the like where the number of product items is large and the product life cycle is short.
[Others]
Hereinafter, inventions related to the embodiments (examples) disclosed in this specification are disclosed as reference inventions.
(1) The information processing system of Reference Invention 1 is
First reproducing means for reproducing first reproduction data including image information;
Recognition means for recognizing an input during reproduction of the first reproduction data;
Generating means for generating sound information based on the input recognized by the recognition means;
Association means for associating the sound information generated by the generation means with the first reproduction data;
It is a summary to provide.
According to this, sound information is generated based on the input during the reproduction of the first reproduction data, and the generated sound information and the first reproduction data are associated with each other. Can be achieved.
(2) The information processing system of Reference Invention 2 is the information processing system of Reference Invention 1 described above.
The gist of the present invention is to provide second reproduction means for reproducing second reproduction data in which the sound information and the first reproduction data are associated with each other.
According to this, the confirmation of the first reproduction data including the sound information (that is, the reproduction data including the generated sound information and the image information) is facilitated by the reproduction of the second reproduction data.
(3) The information processing system of Reference Invention 3 is the information processing system of Reference Invention 1 or Reference Invention 2 described above,
The gist of the association means is that the sound information generated based on the input is associated with the first reproduction data in accordance with the input timing that triggered the generation of the sound information by the generation means.
According to this, it is possible to synchronize the input timing during the reproduction of the first reproduction data and the reproduction timing of the sound information, and it becomes easy to set (determine) the reproduction timing (reproduction position) of the sound information.
(4) The information processing system according to Reference Invention 4 is the information processing system according to any one of Reference Invention 1 to Reference Invention 3,
The gist of the present invention is to provide an adjusting means capable of adjusting the reproduction timing of the sound information associated with the first reproduction data.
According to this, since it is possible to adjust the reproduction timing (reproduction position) of the associated sound information, it is possible to optimize the reproduction timing of the sound information.
(5) The information processing system of Reference Invention 5 is the information processing system of any one of Reference Invention 1 to Reference Invention 4 described above,
Comprising storage means for storing a plurality of sound information;
The generation means acquires sound information corresponding to the input recognized by the recognition means from the sound information stored in the storage means, and generates sound information associated with the first reproduction data. Is the gist.
According to this, since the sound information corresponding to the input is acquired and generated from the plurality of pieces of sound information stored in advance, it is possible to diversify the associated sound information.
(6) The information processing system of Reference Invention 6 is the information processing system of Reference Invention 5 described above,
When the sound information stored in the storage means includes a plurality of sound information corresponding to the input recognized by the recognition means, the generation means determines any one of the plurality of sound information as a predetermined condition. The gist is to specify and acquire based on the above.
According to this, even if there is a plurality of sound information corresponding to the input, any one of the sound information is associated with the first reproduction data, so that the selection of the sound information to be associated is made efficient. Is possible.
(7) The information processing system of Reference Invention 7 is the information processing system of Reference Invention 5 or Reference Invention 6 described above,
Sound information stored in the storage means is managed by classification,
A designation unit capable of designating a classification of sound information to be generated by the generation unit among the classifications;
The generating means acquires sound information corresponding to the input recognized by the recognizing means from the sound information of the classification specified by the specifying means among the sound information stored in the storage means. Is the gist.
According to this, by specifying the classification (type) of the sound information to be associated in advance, the sound information corresponding to the specified classification is generated based on the input during the reproduction of the first reproduction data, and Since it is associated with the first reproduction data, it is possible to improve the efficiency of selection of the sound information to be associated.
(8) An information processing system according to Reference Invention 8 is the information processing system according to any one of Reference Invention 5 to Reference Invention 7,
The gist of the present invention is that the generation means can synthesize the sound information stored in the storage means to generate one sound information corresponding to the input recognized by the recognition means.
According to this, when an input is made during the reproduction of the first reproduction data, even if the sound information matching the input is not stored in the storage means, the sound information stored in the storage means is synthesized. Since one piece of sound information corresponding to the input is generated and associated with the first reproduction data, depending on the content of the input, the sound information is not generated and no association with the first reproduction data is performed as much as possible. It becomes possible to eliminate.
(9) An information processing system according to Reference Invention 9 is the information processing system according to any one of Reference Invention 1 to Reference Invention 8,
The gist of the present invention is that the recognizing means is a sound recognizing means for recognizing a voice spoken during reproduction of the first reproduction data.
According to this, sound information is generated based on the voice uttered during the reproduction of the first reproduction data, and the generated sound information and the first reproduction data are associated with each other. Association with reproduction data can be easily performed by voice input (voice recognition).
(10) The information processing apparatus of Reference Invention 10 is
An information processing apparatus connectable to a reproduction processing apparatus capable of reproducing reproduction data including image information,
Recognizing means for recognizing input during reproduction of reproduction data by the reproduction processing device;
Generating means for generating sound information based on the input recognized by the recognition means;
Association instruction means for instructing the reproduction processing apparatus to associate the sound information generated by the generation means with the reproduction data;
It is a summary to provide.
According to this, since the sound information is generated and associated with the reproduction data based on the input during reproduction of the reproduction data in the reproduction processing apparatus, it is possible to improve the efficiency of the work related to image production.
(11) The information processing apparatus of Reference Invention 11 is the information processing apparatus of Reference Invention 10 described above,
The gist of the present invention is to provide reproduction instruction means for instructing the reproduction processing device to reproduce reproduction data associated with the sound information.
According to this, reproduction data including sound information (that is, reproduction data including generated sound information and image information) can be reproduced by the reproduction processing device. The confirmation work of the reproduction data including becomes easy.
(12) The information processing apparatus of Reference Invention 12 is the information processing apparatus of Reference Invention 10 or Reference Invention 11 described above,
The related instructing means instructs the association between the sound information generated based on the input and the reproduction data in accordance with the input timing that triggered the generation of the sound information by the generating means. To do.
According to this, it becomes possible to synchronize the input timing during the reproduction of the reproduction data and the reproduction timing of the sound information, and the setting (determination) of the reproduction timing (reproduction position) of the sound information becomes easy.
(13) An information processing apparatus according to Reference Invention 13 is the information processing apparatus according to any one of Reference Invention 10 to Reference Invention 12,
Comprising storage means for storing a plurality of sound information;
The generating means acquires sound information corresponding to the input recognized by the recognizing means from sound information stored in the storage means, and generates sound information associated with the reproduction data. And
According to this, since the sound information corresponding to the input is acquired and generated from the plurality of pieces of sound information stored in advance, it is possible to diversify the associated sound information.
(14) The information processing apparatus of Reference Invention 14 is the information processing apparatus of Reference Invention 13 described above,
When the sound information stored in the storage means includes a plurality of sound information corresponding to the input recognized by the recognition means, the generation means determines any one of the plurality of sound information as a predetermined condition. The gist is to specify and acquire based on the above.
According to this, even if there is a plurality of sound information corresponding to the input, since any sound information is associated with the reproduction data, it is possible to improve the efficiency of selecting the associated sound information. It becomes.
(15) The information processing apparatus of Reference Invention 15 is the information processing apparatus of Reference Invention 13 or Reference Invention 14 described above,
Sound information stored in the storage means is managed by classification,
A designation unit capable of designating a classification of sound information to be generated by the generation unit among the classifications;
The generating means acquires sound information corresponding to the input recognized by the recognizing means from the sound information of the classification specified by the specifying means among the sound information stored in the storage means. Is the gist.
According to this, by specifying the classification (type) of the sound information to be associated in advance, the sound information corresponding to the designated classification is generated based on the input during the reproduction of the reproduction data, and the reproduction data Therefore, it is possible to improve the efficiency of selecting sound information to be associated.
(16) An information processing apparatus according to Reference Invention 16 is the information processing apparatus according to any one of Reference Invention 13 to Reference Invention 15,
The gist of the present invention is that the generation means can synthesize the sound information stored in the storage means to generate one sound information corresponding to the input recognized by the recognition means.
According to this, when input is performed during reproduction of reproduction data, even if sound information that matches the input is not stored in the storage means, the sound information stored in the storage means is synthesized. Since one sound information corresponding to the input is generated and associated with the reproduction data, it is possible to eliminate as much as possible that the sound information is not generated and the association with the reproduction data is not performed depending on the content of the input. Become.
(17) An information processing apparatus according to Reference Invention 17 is the information processing apparatus according to any one of Reference Invention 10 to Reference Invention 16,
The gist of the present invention is that the recognizing means is a voice recognizing means for recognizing a voice uttered during reproduction of reproduction data by the reproduction processing device.
According to this, sound information is generated based on the speech uttered during the reproduction of the reproduction data, and the generated sound information is associated with the reproduction data, so that the association between the sound information and the reproduction data is input by voice ( (Speech recognition) can be easily performed.

  DESCRIPTION OF SYMBOLS 10 Information processing system, 20 Playback processing apparatus, 21 Playback part, 22 Editing part, 23 Reading part, 30 Information processing apparatus, 31 Voice recognition part, 32 Search part, 33 Instruction part, 100 CPU, 101 ROM, 102 RAM, 103 Frame buffer memory, 104 image compression / decompression unit, 105 auxiliary storage unit, 106 display unit, 107 operation input unit, 108 audio input unit, 109 sound output unit, 200 editing screen, 201 moving image title display area, 202 playback display area, 203 Timeline display area, 204 Sound edit display area, 205 Voice recognition display area, 210 Genre specification screen, TC timeline cursor, HK waveform, MK mark, IC sound icon.

Claims (2)

Re co means you play including playback data image information,
And recognition means for recognizing input during playback of the previous SL re-raw data,
Generating means for generating sound information based on the input recognized by the recognition means;
And related means for associating the sound information and the prior SL playback data generated by the generating means,
Storage means for storing a plurality of sound information,
The generation means acquires sound information corresponding to the input recognized by the recognition means from the sound information stored in the storage means, and generates sound information associated with the reproduction data. ,
Sound information stored in the storage means is managed by classification,
A specifying unit capable of specifying a classification of sound information to be generated by the generating unit among the classifications;
The generating means acquires sound information corresponding to the input recognized by the recognizing means from the sound information of the classification specified by the specifying means among the sound information stored in the storage means. An information processing system characterized by An information processing apparatus connectable to a reproduction processing apparatus capable of reproducing reproduction data including image information,
Recognizing means for recognizing input during reproduction of reproduction data by the reproduction processing device;
Generating means for generating sound information based on the input recognized by the recognition means;
Association instruction means for instructing the reproduction processing apparatus to associate the sound information generated by the generation means with the reproduction data;
Storage means for storing a plurality of sound information,
The generation means acquires sound information corresponding to the input recognized by the recognition means from the sound information stored in the storage means, and generates sound information associated with the reproduction data. ,
Sound information stored in the storage means is managed by classification,
A specifying unit capable of specifying a classification of sound information to be generated by the generating unit among the classifications;
The generating means acquires sound information corresponding to the input recognized by the recognizing means from the sound information of the classification specified by the specifying means among the sound information stored in the storage means. information processing apparatus said.

Images