JP7509409B2 - Processing device, processing method, and control program - Google Patents

Description

The present disclosure relates to a processing device, a processing method, and a control program.

Patent document 1 proposes a video search device that generates a video database in which video data and location time data are associated based on time, and when a location on a map is specified, plays video data in the vicinity of this location.

JP 2001-290820 A

The inventor has found that there is a need to display video of "points of interest" with specified identification information (e.g., names). For example, there is a need to instantly display video of equipment or areas to be inspected (i.e., points of interest) with specified identification information from video data captured by a camera installed on a railway vehicle.

However, the technology disclosed in Patent Document 1 cannot meet this need and may not be convenient for users.

The objective of the present disclosure is to provide a processing device, a processing method, and a control program that can improve user convenience.

A processing device according to a first aspect is a processing device that records photographed data captured by a movable photographing device,
a recording means for recording the stream from the image capture device as the image capture data at a predetermined frame rate and adding information regarding a capture start time to the image capture data;
an acquired position related information forming means for starting acquisition of the position of the photographing device in synchronization with the photographing start time and acquiring the position of the photographing device at a predetermined time period from the photographing start time, and forming acquired position related information holding a plurality of information units, each information unit including the acquired position and an acquisition time at which the position was acquired;
an association information forming means for forming association information, each of which holds a plurality of information units including the acquisition time and the identification information of the point of interest, based on point of interest management information and the acquisition position related information, the plurality of information units each including an identification information of a point of interest and a point of interest position where the point of interest is located;
a means for forming photographed point information including a plurality of information units corresponding to the plurality of information units of the link information based on the link information and a shooting start time added to the shooting data, each information unit of the photographed point information including a playback time corresponding to a value obtained by subtracting the shooting start time from the acquisition time of the link information and identification information of the attention point;
Equipped with:

A processing method according to a second aspect is a processing method by a processing device that records photographic data photographed by a movable photographing device, comprising:
recording the stream from the image capture device at a predetermined frame rate as the image capture data, and adding information regarding a capture start time to the image capture data;
starting to acquire the position of the photographing device in synchronization with the photographing start time and acquiring the position of the photographing device at a predetermined time interval from the photographing start time, and forming acquired position related information holding a plurality of information units, each information unit including the acquired position and an acquisition time at which the position was acquired;
forming linked information, each of which holds a plurality of information units including the acquisition time and the identification information of the point of interest, based on the point of interest management information and the acquisition position related information, each of which holds a plurality of information units including identification information of a point of interest and a point of interest position where the point of interest is located; and
This includes forming photographed point information including a plurality of information units corresponding respectively to the plurality of information units of the linkage information based on the linkage information and the shooting start time assigned to the shooting data, and each information unit of the photographed point information includes a playback time corresponding to a value obtained by subtracting the shooting start time from the acquisition time of the linkage information, and identification information of the point of interest.

A control program according to a third aspect of the present invention includes a processing device that records photographic data captured by a movable photographing device, the processing device including:
recording the stream from the image capture device at a predetermined frame rate as the image capture data, and adding information regarding a capture start time to the image capture data;
starting to acquire the position of the photographing device in synchronization with the photographing start time and acquiring the position of the photographing device at a predetermined time interval from the photographing start time, and forming acquired position related information holding a plurality of information units, each information unit including the acquired position and an acquisition time at which the position was acquired;
forming linked information, each of which holds a plurality of information units including the acquisition time and the identification information of the point of interest, based on the point of interest management information and the acquisition position related information, each of which holds a plurality of information units including identification information of a point of interest and a point of interest position where the point of interest is located; and
A process is executed that includes forming photographed point information including a plurality of information units corresponding respectively to the plurality of information units of the linkage information based on the linkage information and the shooting start time added to the shooting data, and each information unit of the photographed point information includes a playback time corresponding to a value obtained by subtracting the shooting start time from the acquisition time of the linkage information and identification information of the point of interest.

This disclosure makes it possible to provide a processing device, a processing method, and a control program that can improve user convenience.

FIG. 2 is a block diagram showing an example of a processing device according to the first embodiment. FIG. 13 is a diagram illustrating an example of a recording system according to a second embodiment. FIG. 11 is a diagram illustrating an example of acquired position related information. FIG. 11 is a diagram showing an example of attention point management information. FIG. 11 is a diagram illustrating an example of cooperation information. FIG. 11 is a diagram showing an example of photographed point information. FIG. 4 is a diagram illustrating an example of a user interface displayed on a display device. FIG. 13 is a diagram showing a modified example of the attention point management information. FIG. 2 illustrates an example of a hardware configuration of a processing device.

Hereinafter, the embodiments will be described with reference to the drawings. In the embodiments, the same or equivalent elements are given the same reference numerals, and duplicate descriptions will be omitted.

First Embodiment
Fig. 1 is a block diagram showing an example of a processing device in the first embodiment. The processing device 10 shown in Fig. 1 is a device that records image data captured by a movable image capture device (not shown). In Fig. 1, the processing device 10 has a recording unit 11, an acquisition position related information forming unit 12, a link information forming unit 13, and an imaged point information forming unit 14.

The recording unit 11 receives a stream (e.g., a video or a group of image frames) from a shooting device (not shown). The recording unit 11 records the received stream in a storage unit (not shown) as "shooting data" at a "predetermined frame rate". The above "predetermined frame rate" may be, for example, 15 fps. The recording unit 11 then attaches "information relating to the shooting start time" for the above stream to the above "shooting data". For example, the recording unit 11 may attach the "shooting start time" to the file of the recorded shooting data as the "file name".

The acquired position related information forming unit 12 starts acquiring information about the position of the imaging device (not shown) in synchronization with the above-mentioned shooting start time, and acquires information about the position of the imaging device (not shown) at a "predetermined time period" from the above-mentioned shooting start time. The above-mentioned "predetermined time" may be one second. The acquired position related information forming unit 12 then forms "acquired position related information (acquired position related file)." The "acquired position related information" includes multiple information units, and each information unit includes the position acquired by the acquired position related information forming unit 12 and the "acquisition time" at which this position was acquired.

The linkage information formation unit 13 forms "linkage information (linkage file)" based on the "point of interest management information (point of interest management file)" and the above-mentioned acquisition position related information. The "point of interest management information" holds multiple information units, each of which includes "point of interest identification information" and the position where the point of interest is located (hereinafter sometimes referred to as "point of interest position"). A "point of interest" is, for example, an object or area that is to be checked (confirmed) using video. In other words, a "point of interest" can also be called a "check point" or "inspection point", etc. The "linkage information" holds multiple information units, each of which includes the above-mentioned acquisition time and the above-mentioned point of interest identification information.

The photographed point information forming unit 14 forms "photographed point information" based on the linkage information and the shooting start time added to the shooting data. The "photographed point information" includes multiple information units, which correspond to the multiple information units of the linkage information. Each information unit of the "photographed point information" includes a "playback time" and identification information of the point of interest. The "playback time" corresponds to the value obtained by subtracting the shooting start time from the acquisition time of the linkage information. For example, the photographed point information can be formed by converting the linkage information using the shooting start time.

As described above, according to the first embodiment, the recording unit 11 in the processing device 10 records the stream from the imaging device (not shown) as imaging data at a predetermined frame rate in the storage unit (not shown). The recording unit 11 attaches information about the imaging start time for the stream to the imaging data. The acquisition position related information forming unit 12 starts acquiring information about the position of the imaging device (not shown) in synchronization with the imaging start time, and acquires information about the position of the imaging device (not shown) at a "predetermined time period" from the above-mentioned imaging start time. The acquisition position related information forming unit 12 forms "acquisition position related information (acquisition position related file)". The "acquisition position related information" includes multiple information units, and each information unit includes the position acquired by the acquisition position related information forming unit 12 and the "acquisition time" at which this position was acquired. The linkage information forming unit 13 forms linkage information based on the attention point management information and the acquisition position related information. The linkage information holds multiple information units, each information unit including an acquisition time and attention point identification information. The photographed point information forming unit 14 forms photographed point information based on the linkage information and the shooting start time added to the shooting data. The photographed point information includes multiple information units, which correspond to the multiple information units of the linkage information. Each information unit of the photographed point information includes a playback time and identification information of the point of interest. The playback time corresponds to a value obtained by subtracting the shooting start time from the acquisition time of the linkage information.

This configuration of the processing device 10 makes it possible to form photographed point information that holds information units including the playback time and identification information of the point of interest. By using this photographed point information and specifying the identification information of the point of interest that you want to check, you can easily specify the playback time at which the video of that point of interest will be displayed when the shooting data is played back. This makes it easy to cue up the video of the specified point of interest. This improves user convenience.

The processing device 10 executes the following processing method. This processing method includes recording the stream from the shooting device as shooting data at a predetermined frame rate and attaching information related to the shooting start time to the shooting data, starting acquisition of the position of the shooting device in synchronization with the shooting start time and acquiring the position of the shooting device at a predetermined time interval from the shooting start time, and forming acquired position related information that holds multiple information units, each information unit including the acquired position and the acquisition time at which this position was acquired, forming linkage information that holds multiple information units, each information unit including the acquisition time and identification information of the point of interest, based on the point of interest management information and the acquisition position related information, each information unit including identification information of the point of interest and the point of interest position where the point of interest is located, and forming photographed point information that includes multiple information units corresponding to the multiple information units of the linkage information based on the linkage information and the shooting start time attached to the shooting data, and each information unit of the photographed point information includes a playback time corresponding to a value obtained by subtracting the shooting start time from the acquisition time of the linkage information and identification information of the point of interest.

Second Embodiment
The second embodiment relates to a more specific embodiment.

<Recording system overview>
Fig. 2 is a diagram showing an example of a recording system in the second embodiment. In Fig. 2, the recording system 1 has a photographing device 20, a processing device 30, and a display device 40. Although the manner in which the photographing device 20, the processing device 30, and the display device 40 are used is not limited to this, the following description will be given taking as an example a case in which the photographing device 20, the processing device 30, and the display device 40 are disposed in a railway vehicle.

The imaging device 20 captures images of the area around the railway line (including the tracks) while the train is traveling and sends the captured image stream (e.g., a video or a group of image frames) to the processing device 30.

The processing device 30 receives the stream sent from the image capture device 20. The processing device 30 then stores the received stream as image capture data (i.e., records it). Note that the image capture device 20 and the processing device 30 may be connected by wire or wirelessly.

The display device 40 displays the shooting data stored in the processing device 30.

<Example of processing device configuration>
In FIG. 2 , the processing device 30 has a recording unit 31 , an acquired position related information forming unit 32 , a linkage information forming unit 33 , a photographed point information forming unit 34 , a display control unit 35 , an operation receiving unit 36 , and a memory unit 37 .

The recording unit 31 receives a stream from the imaging device 20, similar to the recording unit 11 of the first embodiment. The recording unit 31 records the received stream in the memory unit 37 as imaging data 37A at a "predetermined frame rate". The recording unit 31 then attaches "information related to the imaging start time" for the above stream to the above "imaging data". For example, the imaging start time is "20190101131500", that is, 13:15:00 on January 1, 2019.

The acquired position related information forming unit 32, like the acquired position related information forming unit 12 of the first embodiment, starts acquiring information related to the position of the imaging device 20 in synchronization with the above-mentioned shooting start time, and acquires information related to the position of the imaging device 20 at a "predetermined time period" from the above-mentioned shooting start time. A GPS reception signal may be used to acquire this position. The acquired position related information forming unit 32 then forms acquired position related information (acquired position related file). The "acquired position related information" includes multiple information units, and each information unit includes a position acquired by the acquired position related information forming unit 32 and the "acquisition time" at which this position was acquired. The acquired position related information forming unit 32 then stores the formed acquired position related information in the storage unit 37 as acquired position related information 37B. FIG. 3 is a diagram showing an example of acquired position related information.

The linkage information forming unit 33, like the linkage information forming unit 13 in the first embodiment, forms linkage information 37D based on the attention point management information 37C and the acquired position related information 37B stored in the memory unit 37.

Figure 4 is a diagram showing an example of point of interest management information. The point of interest management information 37C shown in Figure 4 includes a "facility name" and a "facility location (longitude, latitude)" for each information unit. The "facility name" corresponds to the above-mentioned point of interest identification information, and the "facility location" corresponds to the above-mentioned point of interest location. As described above, it is assumed here that the imaging device 20 and processing device 30 are installed and used in a railway vehicle, so an example of point of interest management information that can be used for the maintenance of railway company facilities is shown.

For example, when the point of interest position of an information unit (hereinafter referred to as the "first information unit") of the point of interest management information 37C is included within a predetermined area based on the acquisition position of an information unit (hereinafter referred to as the "second information unit") of the acquisition position related information 37B, the linkage information forming unit 33 adds an information unit including the point of interest identification information of the first information unit and the acquisition time of the second information unit to the linkage information 37D. For example, the equipment position of the top information unit of the point of interest management information 37C shown in FIG. 4 and the acquisition position (longitude, latitude) of the top information unit of the acquisition position related information 37B shown in FIG. 3 match. Therefore, the linkage information forming unit 33 adds an information unit including the acquisition time of the top information unit of the acquisition position related information 37B and the equipment name of the top information unit of the point of interest management information 37C to the linkage information 37D. The linkage information forming unit 33 may add an information unit including the acquisition position of the second information unit in addition to the attention point identification information of the first information unit and the acquisition time of the second information unit to the linkage information 37D. FIG. 5 is a diagram showing an example of linkage information.

The photographed point information forming unit 34, like the photographed point information forming unit 14 of the first embodiment, forms photographed point information 37E based on the above-mentioned linkage information and the shooting start time attached to the above-mentioned shooting data. The photographed point information 37E includes multiple information units, and these multiple information units correspond to the multiple information units of the linkage information 37D, respectively. In addition, each information unit of the photographed point information 37E includes a "playback time" and identification information of the attention point. The "playback time" corresponds to a value obtained by subtracting the shooting start time from the acquisition time of the linkage information 37D. In other words, the photographed point information forming unit 34 can form the photographed point information 37E by calculating a value obtained by subtracting the shooting start time from the acquisition time of the linkage information 37D and converting the linkage information 37D using the calculated value. FIG. 6 is a diagram showing an example of photographed point information.

When the operation reception unit 36 receives a facility name (identification information of a point of interest) specified by a user operation, it outputs the facility name to the display control unit 35.

When the display control unit 35 receives the equipment name from the operation reception unit 36, it searches the photographed point information 37E and acquires the playback time associated with the received equipment name in the photographed point information 37E. Then, the display control unit 35 executes control to display the image of the frame corresponding to the acquired playback time on the display device 40. For example, the display control unit 35 displays the image of the frame having the frame number of "(playback time x frame rate) + 1" on the display device 40. The display control unit 35 may automatically play the image of the shooting data from the frame corresponding to the acquired playback time, or may play the image of the shooting data from the frame corresponding to the acquired playback time when the user performs a playback operation on the operation reception unit 36. The display control unit 35 may display the image of the shooting data and the identification information of the attention point associated with the playback time of the displayed image in the photographed point information 37E. FIG. 7 is a diagram showing an example of a user interface displayed on the display device.

The display control unit 35 may read out map data 37F from the memory unit 37 and display a map including the facility location that is associated with the facility name displayed on the display device 40 in the photographed point information 37E, and may also display this facility location on this map.

<Other embodiments>
<1> The attention point management information described in the second embodiment may be modified as follows. That is, the attention point management information may include an "area name" including the location of the equipment in addition to the "facility name" and "facility location (longitude, latitude)". When this attention point management information is used, the photographed point information becomes, for example, information as shown in FIG. 8. By using this photographed point information, the user can specify an area name and check the video of the equipment that exists in the area corresponding to this area name.

<2> In the second embodiment, the explanation is given on the premise that the positions that change with the actual movement of the imaging device 20 and the processing device 30 are acquired and a position file is formed. For example, a railway company may form a position file that holds multiple information units, each information unit including the scheduled position and scheduled time of a railway vehicle, based on the scheduled running time of the railway vehicle.

<3> FIG. 9 is a diagram showing an example of the hardware configuration of a processing device. In FIG. 9, the processing device 100 has a processor 101 and a memory 102. The processor 101 may be, for example, a microprocessor, an MPU (Micro Processing Unit), or a CPU (Central Processing Unit). The processor 101 may include multiple processors. The memory 102 is configured by a combination of volatile memory and non-volatile memory. The memory 102 may include storage located away from the processor 101. In this case, the processor 101 may access the memory 102 via an I/O interface not shown.

The processing devices 10 and 30 of the first and second embodiments may each have the hardware configuration shown in FIG. 9. The recording units 11 and 31, the acquired position related information forming units 12 and 32, the linkage information forming units 13 and 33, the photographed point information forming units 14 and 34, the display control unit 35, and the operation receiving unit 36 of the processing devices 10 and 30 of the first and second embodiments may be realized by the processor 101 reading and executing a program stored in the memory 102. The storage unit 37 may be realized by the memory 102. The program can be stored using various types of non-transitory computer readable media and supplied to the processing devices 10 and 30. Examples of non-transitory computer readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives) and magneto-optical recording media (e.g., magneto-optical disks). Furthermore, examples of non-transitory computer readable media include CD-ROMs (Read Only Memory), CD-Rs, and CD-R/Ws. Further, examples of non-transitory computer-readable media include semiconductor memories. Semiconductor memories include, for example, mask ROMs, programmable ROMs (PROMs), erasable PROMs (EPROMs), flash ROMs, and random access memories (RAMs). The program may also be provided to the processing devices 10 and 30 by various types of transitory computer-readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer-readable media can provide the program to the processing devices 10 and 30 via wired communication paths such as electrical wires and optical fibers, or wireless communication paths.

The present invention has been described above with reference to the embodiment, but the present invention is not limited to the above. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present invention within the scope of the invention.

REFERENCE SIGNS LIST 1 Recording system 10 Processing device 11 Recording section 12 Acquired position related information forming section 13 Linkage information forming section 14 Photographed point information forming section 20 Photographing device 30 Processing device 31 Recording section 32 Acquired position related information forming section 33 Linkage information forming section 34 Photographed point information forming section 35 Display control section 36 Operation reception section 37 Memory section 37A Photographing data 37B Acquired position related information 37C Point of interest management information 37D Linkage information 37E Photographed point information 37F Map data 40 Display device

Claims (8)

A processing device that records photographic data captured by a movable photographing device,
a recording means for recording the stream from the image capture device as the image capture data at a predetermined frame rate and attaching information relating to a capture start time to the image capture data;
an acquired position related information forming means for starting acquisition of the position of the photographing device in synchronization with the photographing start time and acquiring the position of the photographing device at a predetermined time period from the photographing start time, and forming acquired position related information holding a plurality of information units, each information unit including the acquired position and an acquisition time at which the position was acquired;
an association information forming means for forming association information, each of which holds a plurality of information units including the acquisition time and the identification information of the point of interest, based on point of interest management information and the acquisition position related information, the plurality of information units each including an identification information of a point of interest and a point of interest position where the point of interest is located;
a means for forming photographed point information including a plurality of information units corresponding to the plurality of information units of the link information based on the link information and a shooting start time added to the shooting data, each information unit of the photographed point information including a playback time corresponding to a value obtained by subtracting the shooting start time from the acquisition time of the link information and identification information of the attention point;
A processing device comprising: when a position of the point of interest of a first information unit of the point of interest management information is included within a predetermined area based on the acquisition position of a second information unit of the acquisition position related information, the linking information forming means adds an information unit including identification information of the point of interest of the first information unit and the acquisition time of the second information unit to the linking information.
The processing apparatus of claim 1. a display control means for displaying a video of the photographed data from a frame corresponding to a playback time associated with identification information of a designated attention point in the photographed point information;
The processing device according to claim 1 or 2. the display control means displays the image of the photographed data, and displays identification information of the attention point associated with the photographed point information on the playback time of the displayed image.
The processing device according to claim 3 . Each information unit of the link information includes the acquired position in addition to the acquisition time and the identification information of the point of interest,
Each information unit of the photographed point information includes the acquired position in addition to the playback time and the identification information of the attention point,
the display control means displays a map including a position associated with the identification information of the displayed point of interest in the link information, and displays the position on the map.
The processing apparatus according to claim 4. The recording means assigns the shooting start time as a file name to the file of the recorded shooting data.
The processing device according to any one of claims 1 to 5. A processing method for a processing device that records photographing data photographed by a movable photographing device, comprising:
recording the stream from the image capture device at a predetermined frame rate as the image capture data, and adding information regarding a capture start time to the image capture data;
starting to acquire the position of the photographing device in synchronization with the photographing start time and acquiring the position of the photographing device at a predetermined time interval from the photographing start time, and forming acquired position related information holding a plurality of information units, each information unit including the acquired position and an acquisition time at which the position was acquired;
forming linked information, each of which holds a plurality of information units including the acquisition time and the identification information of the point of interest, based on the point of interest management information and the acquisition position related information, each of which holds a plurality of information units including identification information of a point of interest and a point of interest position where the point of interest is located; and
forming photographed point information including a plurality of information units corresponding to the plurality of information units of the link information based on the link information and a shooting start time added to the shooting data, each information unit of the photographed point information including a playback time corresponding to a value obtained by subtracting the shooting start time from the acquisition time of the link information and identification information of the attention point;
Processing method. A processing device that records the photographed data photographed by the movable photographing device,
recording the stream from the image capture device at a predetermined frame rate as the image capture data, and adding information regarding a capture start time to the image capture data;
starting to acquire the position of the photographing device in synchronization with the photographing start time and acquiring the position of the photographing device at a predetermined time interval from the photographing start time, and forming acquired position related information holding a plurality of information units, each information unit including the acquired position and an acquisition time at which the position was acquired;
forming linked information, each of which holds a plurality of information units including the acquisition time and the identification information of the point of interest, based on the point of interest management information and the acquisition position related information, each of which holds a plurality of information units including identification information of a point of interest and a point of interest position where the point of interest is located; and
Executing a process including forming photographed point information including a plurality of information units corresponding to the plurality of information units of the link information based on the link information and a shooting start time added to the shooting data, each information unit of the photographed point information including a playback time corresponding to a value obtained by subtracting the shooting start time from the acquisition time of the link information and identification information of the attention point.
Control program.

Images