CN110309245B

CN110309245B - Data management method and apparatus

Info

Publication number: CN110309245B
Application number: CN201910319266.5A
Authority: CN
Inventors: 孙小倩; 塞巴斯蒂安·万德特; 伊鹏飞
Original assignee: Beihang University
Current assignee: Beihang University
Priority date: 2019-04-19
Filing date: 2019-04-19
Publication date: 2021-08-31
Anticipated expiration: 2039-04-19
Also published as: CN110309245A

Abstract

The data management method and device provided by the present invention include: acquiring query conditions input by a user, and acquiring target data blocks corresponding to the query conditions from metadata, wherein the metadata includes the correspondence between the query conditions and the data blocks, and the target data blocks are included in the metadata. The data block includes multiple ADS-B data of ADS-B, and from the target data block, the target ADS-B data is obtained. Find the target data block according to the query conditions, and obtain the target ADS‑B data from the target data block, thereby improving the data access speed.

Description

Data management method and device

Technical Field

The present invention relates to the field of data management technologies, and in particular, to a data management method and apparatus.

Background

During air transportation, along with the increasing demand of air transportation efficiency and safety, a great deal of traffic data of an airplane needs to be collected and analyzed. The introduction of Automatic Dependent Surveillance-Broadcast (ADS-B) enabled researchers to acquire large amounts of traffic data (ADS-B data) with very high spatial coverage and temporal frequency.

For a large amount of ADS-B data, lossless compression is usually adopted to reduce the occupied space of the data according to the regularity and similarity of the ADS-B data, however, in this way, when query response is performed, the compressed packets of the entire ADS-B data need to be decompressed, thereby resulting in low data access speed.

Disclosure of Invention

In order to solve the problems in the prior art, the invention provides a data management method and a data management device, when target Broadcast Automatic Dependent Surveillance-Broadcast (ADS-B) data is obtained, the target ADS-B data can be obtained only according to a target data block, and therefore, the access speed is improved.

In a first aspect, an embodiment of the present invention provides a data management method, including:

acquiring a query condition input by a user;

acquiring a target data block corresponding to the query condition from metadata, wherein the metadata comprises the corresponding relation between the query condition and the data block, and the target data block comprises a plurality of broadcast type automatic dependent surveillance (ADS-B) data;

and acquiring target ADS-B data from the target data block.

In one possible design, the method further includes:

acquiring ADS-B data in the ith preset period, wherein each ADS-B data comprises an airplane identifier, latitude, longitude, altitude, time, origin and destination, and i is an integer greater than or equal to 1;

acquiring a latitude value list in an ith preset period and a longitude value list in the ith preset period, wherein the latitude value list in the ith preset period is the latitude value list of ADS-B data in the ith preset period, and the longitude value list in the ith preset period is the longitude value list of the ADS-B data in the ith preset period;

sorting the latitude value list in the ith preset time period, and extracting X-1 latitude values at the distributed indexes such as X-1, wherein X is an integer greater than or equal to 2; and

sorting the longitude value list in the ith preset period, and extracting X-1 longitude values at distributed indexes such as X-1;

and dividing the ADS-B data in the ith preset period into X X data blocks according to the X-1 longitude values and the X-1 latitude values extracted in the ith preset period.

In one possible design, the method further includes:

and compressing ADS-B data in the X data blocks according to columns.

In one possible design, the method further includes:

acquiring ADS-B data in an (i + 1) th preset time period;

acquiring a latitude value list in an (i + 1) th preset period and a longitude value list in an (i + 1) th preset period, wherein the latitude value list in the (i + 1) th preset period is the latitude value list of ADS-B data in the (i + 1) th preset period, and the longitude value list in the (i + 1) th preset period is the longitude value list of the ADS-B data in the (i + 1) th preset period;

sorting the latitude value list in the (i + 1) th preset time period, and extracting X-1 latitude values at the distributed indexes such as X-1; and

sorting the longitude value list in the (i + 1) th preset time period, and extracting X-1 longitude values at the distributed indexes such as X-1;

judging whether a kth preset time interval exists in the first i preset time intervals, wherein the similarity between the X-1 longitude values and the X-1 latitude values extracted in the kth preset time interval and the X-1 longitude values and the X-1 latitude values extracted in the (i + 1) th preset time interval is smaller than a similarity threshold, and k is an integer which is greater than or equal to 1 and is smaller than or equal to i;

if the k preset time interval exists in the first i preset time intervals, dividing ADS-B data in the (i + 1) th preset time interval into X X data blocks by using X-1 longitude values and X-1 longitude values extracted in the k preset time interval.

In one possible design, the method further includes:

if the k-th preset time interval does not exist in the first i preset time intervals, dividing ADS-B data in the (i + 1) th preset time interval into X X data blocks by using X-1 longitude values and X-1 latitude values extracted in the (i + 1) th preset time interval.

In one possible design, the query condition includes at least one of the following parameters:

time period, latitude and longitude ranges, aircraft identifier, origin, and destination.

In one possible design, the method further includes:

pre-establishing a corresponding relation between the query condition and the data block;

and storing the corresponding relation in the metadata.

In one possible design, the obtaining target ADS-B data from the data block includes:

acquiring a target data block where target ADS-B data is located according to the query condition;

and decompressing at least one column in the target data block according to user operation, and acquiring the target ADS-B data from the decompressed at least one column according to the query condition.

In one possible design, the similarity threshold is 100, the number of blocks X is 9, and the predetermined time period is 30 minutes.

In a second aspect, an embodiment of the present invention provides a data management apparatus, including:

the acquisition module is used for acquiring the query conditions input by the user;

the obtaining module is further configured to obtain a data block corresponding to the query condition from metadata, where the metadata includes a correspondence between the query condition and the data block, and the target data block includes multiple pieces of broadcast-based automatic dependent surveillance (ADS-B) data;

the obtaining module is further configured to obtain target ADS-B data from the target data block.

In a third aspect, an embodiment of the present invention provides an electronic device, including:

a processor;

a memory; and

a computer program;

wherein the computer program is stored in the memory and configured to be executed by the processor, the computer program comprising instructions for performing the method of any of the first aspects.

In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and the computer program causes a server to execute the method according to any one of the first aspect.

The invention provides a data management method and a device, wherein the method comprises the following steps: acquiring a query condition input by a user, acquiring a target data block corresponding to the query condition from metadata, wherein the metadata comprises the corresponding relation between the query condition and the data block, the target data block comprises a plurality of broadcast type automatic dependent surveillance ADS-B data, and the target ADS-B data is acquired from the target data block. And finding a target data block according to the query condition, and acquiring target ADS-B data from the target data block, thereby improving the data access speed.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.

FIG. 1 is a schematic diagram of an ADS-B system according to an embodiment of the present invention;

FIG. 2 is a schematic illustration of a flight path of an a4 aircraft according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of visualizing ADS-B data according to an embodiment of the present invention;

fig. 4 is a flowchart illustrating a data management method according to an embodiment of the present invention;

FIG. 5 is a flowchart illustrating a data management method according to another embodiment of the present invention;

FIG. 6 shows a blocking result of ADS-B data according to an embodiment of the present invention;

FIG. 7 is a flowchart illustrating a data management method according to another embodiment of the present invention;

FIG. 8 is a diagram illustrating a data management apparatus according to an embodiment of the present invention;

fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

First, terms of art related to the present invention will be explained:

the ADS-B can automatically acquire parameters from related airplanes without manual operation or inquiry, and broadcast the position, altitude, speed, course, identification number and other information of the airplanes to other airplanes or ground stations so as to provide a controller to monitor the state of the airplanes. ADS-B not only overcomes some problems of the traditional radar monitoring means, but also has the advantages of high precision, high update rate, wide application range, low construction and maintenance cost of ground equipment and the like.

Fig. 1 is a schematic diagram of an ADS-B system according to an embodiment of the present invention. As shown in fig. 1, the ADS-B system provided in this embodiment includes an aircraft a, an aircraft B, and ground equipment, where the aircraft a is installed with an ADS-B transmitter for transmitting ADS-B data to the aircraft B and the ground equipment, and the aircraft B and the ground equipment are both installed with ADS-B receivers for receiving the ADS-B data transmitted by the aircraft a.

The ADS-B data contains rich flight status information, such as the receiving time t, the aircraft identifier icao, the longitude lon, the latitude lat, and the altitude alt. One ADS-B data can be expressed as:

p＝<t，lon，lat，alt，icao，f₁，f₂，...，f_i>

f₁，f₂，...，f_iother flight status information is shown, and only the departure place and the destination are considered in the embodiment of the invention.

Further, the ADS-B dataset may be denoted as D ═ p₁，p₂，...，p_nWhere D denotes the ADS-B dataset, p₁，p₂，...，p_nRepresenting ADS-B data.

Illustratively, an ADS-B data set includes 30 ADS-B data, denoted as D ═ p₁，p₂，...，p₃₀And (c) the step of (c) in which,the 30 ADS-B data are respectively expressed as:

p₁＝<1,0.5,0.5,9,a1,c1,c2>、p₂＝<1,9.0,1.0,0,a2,c4,c5>；

_p3＝<1,3.0,7.0,9,a3,c3,c6>、p₄＝<1,6.0,2.0,0,a4,c8,c9>；

_p5＝<1,9.0,9.0,0,a5,c5,c4>、p₆＝<2,1.0,1.0,9,a1,c1,c2>；

p₇＝<2,9.0,2.0,3,a2,c4,c5>、p₈＝<2,4.0,6.0,4,a3,c3,c6>；

p₉＝<2,6.1,3.0,2,a4,c8,c9>、p₁₀＝<2,9.0,8.0,3,a5,c5,c4>；

p₁₁＝<3,1.5,1.5,5,a1,c1,c2>、p₁₂＝<3,9.0,4.0,9,a2,c4,c5>；

p₁₃＝<3,5.0,5.0,0,a3,c3,c6>、p₁₄＝<3,6.0,4.0,8,a4,c8,c9>；

p₁₅＝<3,8.8,6.0,9,a5,c5,c4>、p₁₆＝<4,2.0,2.0,3,a1,c1,c2>；

p₁₇＝<4,9.0,6.0,9,a2,c4,c5>、p₁₈＝<4,5.0,5.0,0,a3,c6,c7>；

p₁₉＝<4,6.2,5.0,9,a4,c8,c9>、p₂₀＝<4,8.8,4.0,9,a5,c5,c4>；

p₂₁＝<5,2.4,1.4,1,a1,c1,c2>、p₂₂＝<5,9.0,8.0,3,a2,c4,c5>；

p₂₃＝<5,4.0,4.0,3,a3,c6,c7>、p₂₄＝<5,6.3,7.0,9,a4,c8,c9>；

p₂₅＝<5,9.0,2.0,3,a5,c5,c4>、p₂₆＝<6,2.5,1.5,0,a1,c1,c2>；

P₂₇＝<6,9.0,9.0,0,a2,c4,c5>、p₂₈＝<6,3.0,3.0,9,a3,c6,c7>；

p₂₉＝<6,6.4,8.0,9,a4,c8,c9>、p₃₀＝<6,9.0,1.0,0,a5,c5,c4>。

with p₁＝<1,0.5,0.5,9,a1,c1,c2>For the purpose of illustration, p₁ADS-B data has a time of 1, longitude of 0.5, latitude of 0.5, altitude of 9, aircraft identifier a1, origin c1, destination c 2.

As can be seen, the ADS-B dataset includes flight status information for 5 airplanes a1, a2, a3, a4, and a5 at 6 times t 1, t 2, t 3, t 4, t 5, and t 6.

Fig. 2 is a schematic view of a flight trajectory of an a4 aircraft according to an embodiment of the present invention, and as shown in fig. 2, for an aircraft a4, longitude and latitude coordinates at 6 times, t-1, t-2, t-3, t-4, t-5, and t-6, are connected to obtain a flight trajectory of an aircraft a 4.

Fig. 3 is a schematic diagram of visualized ADS-B data according to an embodiment of the present invention, as shown in fig. 3, the positions of the airplanes are visually represented at time t-1 and time t-4, so as to facilitate observing the positions of the airplanes at time t-1 and time t-4. With a plurality of aircraft a (3 aircraft a shown in fig. 1), as the amount of ADS-B data increases, the large amount of ADS-B data provides opportunities for data analysis on aircraft B and ground equipment, and also presents administrative challenges, particularly on storing data and querying data. Unprocessed raw ADS-B data typically requires a large amount of storage space and slow query response speed. For the problem, two data management methods are provided at present, one is to reduce the occupied amount of the storage space by utilizing the regularity and the similarity of data and adopting a lossless compression mode, however, in the method, the whole data set needs to be decompressed during query response, so that the query response is slow; the other method is to use the solution of the retrieval problem as a starting point, obtain high-efficiency query response capability by rearranging data, establishing indexes and the like, but the established index file is too large and occupies a large storage space. For example, in the query process, when querying at a time, all ADS-B data belonging to the time need to be retrieved from the ADS-B dataset, taking the ADS-B dataset as an example, if t is set to 4, the returned result is p₁₆＝<4,2.0,2.0,3,a1,c1,c2>、p₁₇＝<4,9.0,6.0,9,a2,c4,c5>、p₁₈＝<4,5.0,5.0,0,a3,c6,c7>、p₁₉＝<4,6.2,5.0,9,a4,c8,c9>、p₂₀＝<4,8.8,4.0,9,a5,c5,c4>Therefore, when the query response is performed, the target ADS-B data needs to be searched from the whole data set, which results in slow query response.

In view of the above problems, embodiments of the present invention provide a data management method, and the following detailed description will explain technical solutions of the present invention with specific embodiments. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments.

Fig. 4 is a flowchart illustrating a data management method according to an embodiment of the present invention. The execution subject of this embodiment may be an electronic device, and the electronic device may be the airplane B in the embodiment shown in fig. 1, or may also be a ground device shown in fig. 1, which is not particularly limited in this embodiment. As shown in fig. 2, the method includes:

s101, acquiring a query condition input by a user.

And S102, in the metadata, inquiring and acquiring a target data block corresponding to the inquiry condition.

The following is an exemplary description of the data block.

Suppose a set of ADS-B data sequences is block ═ p₁，...，p_m]M is an integer greater than 1, if p₁，...，p_mArbitrary data p in (1)_iSatisfy lon₁≤p_i.lon＜lon₂,lat₁≤p_i.lat＜lat₂And t is₁≤p_i.t＜t₂And arbitrary data p_iThe data sequence is sorted according to the airplane identifier icao, and then sorted according to the time period t, so that similar data in subsequent column compression after sorting are placed at adjacent positions, and the adjacent data are similar when the data are compressed according to the columns, so that the compression efficiency can be improved, and the data sequence is called as a data block. Wherein p is_i.lon、p_i.lon、p_iT represents data p, respectively_iLongitude, latitude and time of day, lon₁、lon₂Is expressed byThreshold value of degree, lat₁、lat₂Indicating a threshold value of latitude, t₁、t₂A threshold value representing a time period.

In this embodiment, the electronic device obtains a query condition input by a user, and the metadata includes a corresponding relationship between the query condition and a data block, so that a corresponding target data block can be obtained according to the query condition, the number of the target data block is at least one, and each target data block includes a plurality of ADS-B data.

Optionally, the query condition includes at least one of the following parameters: time period, latitude and longitude ranges, aircraft identifier, origin, and destination. The corresponding relationship between the query condition and the data block in the metadata may be stored in the form of tables 1 to 3, and may also be stored in the form of text, which is not limited in this embodiment.

Table 1 is a correspondence between an aircraft identifier and a data block, table 2 is a correspondence between a time period and a data block, and table 3 is a correspondence between a latitude and longitude range and a data block, and as shown in tables 1, 2, and 3, a corresponding data block can be determined according to the aircraft identifier, the time period, and the latitude and longitude range. In addition, the correspondence between the origin and the destination and the data block is similar to the correspondence between the airplane identifier and the data block in table 1, and is not described herein again.

TABLE 1

Aircraft identifier	Data block ID
		a1
	1
		a2	2，4
a3	3
		a4	1，2，3

TABLE 2

Lower limit of time interval	Upper limit of time period	Data block ID
			1	3	1，2，3，4

TABLE 3

Those skilled in the art need to understand that when the query condition is a time period, the query condition is mainly used for a flight tracking system, airplane state display or air traffic flow management and the like; when the query condition is a latitude and longitude range, the method is very useful for the visualization of the flight state in the area or the evaluation of the complexity of the sector of the area; the query condition is an airplane identifier and is used for route optimization, airplane risk avoidance research and the like; the query conditions are an origin and a destination, and are used for researches such as delay prediction and aviation network efficiency.

S103, acquiring target ADS-B data from the target data block.

In this embodiment, the ADS-B data in each data block is compressed by columns, for example, the ADS-B data includes an airplane identifier, latitude, longitude, time, origin, and destination, and the ADS-B data block is compressed by columns according to the airplane identifier, latitude, longitude, time, origin, and destination, and the ADS-B data block includes six compressed columns. When target ADS-B data is acquired from a target data block, firstly, the target data block where the target ADS-B data is located is acquired from metadata according to query conditions, then, at least one column in the target data block is decompressed according to user operation, specifically, at least one column in the target data block can be decompressed according to actual requirements, and the target ADS-B data is acquired from the decompressed at least one column according to the query conditions.

According to the data management method provided by the embodiment of the invention, the query condition input by the user is obtained, the target data block corresponding to the query condition is obtained from the metadata, the metadata comprises the corresponding relation between the query condition and the data block, the target data block comprises a plurality of ADS-B data, and the target ADS-B data is obtained from the target data block. And finding a target data block according to the query condition, and acquiring target ADS-B data from the target data block, thereby improving the data access speed.

Fig. 5 is a flowchart illustrating a data management method according to another embodiment of the present invention. Before the foregoing embodiment of fig. 4, the data management method further includes:

s201, obtaining ADS-B data in the ith preset period, wherein each ADS-B data comprises information such as an airplane identifier, latitude, longitude, time, a place of origin, a destination and the like, and i is an integer greater than or equal to 1.

In consideration of load balance, in order to achieve substantially equal quantity of ADS-B data in each data block, experiments show that the total number of ADS-B data at different times is substantially constant in the global range, and the ADS-B data are not uniformly distributed in space due to flight peaks and night forbidden flights in different areas. Therefore, the present embodiment is uniform for the division of the period, and mainly considers the division method in the latitude and longitude. In this embodiment, taking any ith uniformly divided period as an example, ADS-B data in the ith preset period is acquired.

S202, acquiring a latitude value list in the ith preset period and a longitude value list in the ith preset period.

The latitude value list in the ith preset period is a latitude value list of ADS-B data in the ith preset period, and the longitude value list is a longitude value list of the ADS-B data in the ith preset period.

S203, sorting the latitude value list in the ith preset time period, and extracting X-1 latitude values at the distributed indexes such as X-1, wherein X is an integer greater than or equal to 2.

In this embodiment, the latitude value lists in the ith preset time period may be sorted from small to large, and X-1 latitude values at the distribution index, such as X-1, where X is an integer greater than or equal to 2, are extracted, and specifically, the sorted latitude value list in the ith preset time period is selected as the first latitude value list

Individual latitude value lat₁，lat₂，...，lat_X-1Symbol of

Indicating a rounding down.

S204, sorting the longitude value list in the ith preset period, and extracting X-1 longitude values at the distributed indexes such as X-1.

In this embodiment, the longitude value lists in the ith preset period may be sorted from small to large, and the X-1 longitude values at the distributed indexes such as X-1 are extracted, specifically, the sorted longitude value list in the ith preset period is selected

A longitude value lon₁，lon₂，...，lon_X-1。

S205, dividing ADS-B data in the ith preset period into X X data blocks according to the X-1 longitude values and the X-1 latitude values extracted in the ith preset period.

Assuming that N ADS-B data exist in the ith preset period, taking X-1 longitude values and X-1 latitude values extracted in the ith preset period as a cutting plane, and uniformly dividing the N ADS-B data into X X data blocks.

For example, if the preset time period is 3 for 30 ADS-B data in the embodiment of fig. 1, the ADS-B data are divided into two groups, p₁，p₂，...，p₁₅And p₁₆，p₁₇，...，p₃₀The number of ADS-B data per group is 15.

To form a1 st group of data p₁，p₂，...，p₁₅For example, if the data block is divided into 4 data blocks, N is 15 and X is 2.

p₁，p₂，...，p₁₅The list of longitude values formed is:

list_lon＝[0.5，9.0，3.0，6.0，9.0，1.0，9.0，4.0，6.1，9.0，1.5，9.0，5.0，6.1，8.8]

p₁，p₂，...，p₁₅the list of latitude values formed is:

list_lat＝[0.5，1.0，7.0，2.0，9.0，1.0，2.0，6.0，3.0，8.0，1.5，4.0，5.0，4.0，6.0]

the ordered list of longitude values is:

sorted_list_lon＝[0.5，1.0，1.5，3.0，4.0，5.0，6.0，6.1，6.1，8.8，9.0，9.0，9.0，9.0，9.0]

the sorted list of latitude values is:

sorted_list_lat＝[0.5，1.0，1.0，1.5，2.0，2.0，3.0，4.0，4.0，5.0，6.0，6.0，7.0，8.0，9.0]

then, respectively select the first

Each longitude value is 6.0 and each latitude value is 3.0, 15 data can be divided into 4 data blocks, namely a data block1, a data block2 and dataBlock3, data block4, wherein block1 includes ADS-B data p₁、p₄、p₆、p₁₁The block2 comprises ADS-B data p₂、p₇、p₉The block3 comprises ADS-B data p₃、p₈、p₁₃、p₁₄The block4 comprises ADS-B data p₅、p₁₀、p₁₂、p₁₅。

block₁＝[p₁，p₄，p₆，p₁₁]

block₂＝[p₂，p₇，p₉]

block₃＝[p₃，p₈，p₁₃，p₁₄]

block₄＝[p₅，p₁₀，p₁₂，p₁₅]

P is described below with reference to FIG. 6₁，p₂，...，p₁₅Specifically, fig. 6 is a block result of ADS-B data according to an embodiment of the present invention, and as shown in fig. 6, 15 data blocks are divided into 4 data blocks by using a longitude value of 6.0 and a latitude value of 3.0 as cut planes.

And S206, compressing ADS-B data in the X data blocks according to the columns.

In this embodiment, after dividing the ADS-B data in the ith preset period into X × X data blocks, the ADS-B data in the X × X data blocks are compressed by columns, and the ADS-B data in each data block is compressed by columns, so that efficient data storage can be obtained. Therefore, each data block includes a plurality of compressed columns, which are an airplane identifier, a latitude, a longitude, a time of day, an origin, and a destination. The following description will exemplarily take the data block1 and the data block2 in table 4 as examples.

Table 4 shows the result of column-by-column compression, and as shown in table 4, each of the data block1 and the data block2 includes a plurality of compressed columns, which are time, longitude, and latitude, respectively, and each compressed column includes corresponding compressed data.

TABLE 4

After S206, the method further includes:

pre-establishing a corresponding relation between a query condition and a data block;

the correspondence is saved in the metadata.

Since the query condition includes at least one of the following parameters: the time period, the latitude and longitude range, the aircraft identifier, the origin, and the destination may be stored in the metadata, and specific storage manners may refer to tables 1 to 3 in the above embodiments.

The data management method provided by this embodiment obtains ADS-B data in an ith preset period, where each ADS-B data includes at least one of an aircraft identifier, latitude, longitude, time, origin, and destination, i is an integer greater than or equal to 1, obtains a list of latitude values in the ith preset period and a list of longitude values in the ith preset period, where the list of latitude values in the ith preset period is the list of latitude values of ADS-B data in the ith preset period, and the list of longitude values in the ith preset period is the list of longitude values of ADS-B data in the ith preset period, sorts the list of latitude values in the ith preset period, extracts X-1 latitude values at an index distributed equally by X-1, and X is an integer greater than or equal to 2, sorting the longitude value list in the ith preset period, extracting X-1 longitude values at the distributed index positions such as X-1, dividing ADS-B data in the ith preset period into X data blocks according to the X-1 longitude values and the X-1 latitude values extracted in the ith preset period, and compressing the ADS-B data in the X data blocks according to columns. The data of the same flight and similar latitude and longitude are divided together, which is beneficial to accelerating the access speed, because the data blocks related to the query can be selectively accessed, and in addition, because the similar content between the ADS-B data in each data block, the ADS-B data in the data block in the ith preset time period is compressed according to columns, and the efficient data storage can be obtained.

Fig. 7 is a flowchart illustrating a data management method according to another embodiment of the present invention. On the basis of the embodiment of fig. 5, the data management method further includes:

s301, ADS-B data in the (i + 1) th preset time period are acquired.

S302, a latitude value list in the (i + 1) th preset time period and a longitude value list in the (i + 1) th preset time period are obtained.

The latitude value list in the (i + 1) th preset period is the latitude value list of ADS-B data in the (i + 1) th preset period, and the longitude value list in the (i + 1) th preset period is the longitude value list of ADS-B data in the (i + 1) th preset period.

S303, sorting the latitude value list in the (i + 1) th preset time period, and extracting X-1 latitude values at the distributed indexes such as X-1.

S304, sorting the longitude value list in the (i + 1) th preset time period, and extracting X-1 longitude values at the distributed indexes such as X-1.

The implementation processes of S301-S304 are similar to those of S201-S204, and are not described herein again.

S305, judging whether a k-th preset time interval exists in the previous i preset time intervals, wherein the similarity between the X-1 longitude values and the X-1 latitude values extracted in the k-th preset time interval and the X-1 longitude values and the X-1 latitude values extracted in the (i + 1) -th preset time interval is smaller than a similarity threshold, and k is an integer which is greater than or equal to 1 and less than or equal to i.

S306, if the k preset time interval exists in the first i preset time intervals, dividing ADS-B data in the (i + 1) th preset time interval into X X data blocks by using the X-1 longitude values and the X-1 longitude values extracted in the k preset time interval.

The similarity is the sum of absolute values of differences between every two of the longitude value list and the latitude value list at the X-1 equal distribution index of the kth preset time period and the longitude value list and the latitude value list at the X-1 equal distribution index of the (i + 1) th preset time period, and the following formula is shown:

is the X longitude value in the (i + 1) th preset time period, wherein X is any integer from 1 to X-1;

the x-th latitude value in the (i + 1) -th preset time period;

is the x longitude value in the k preset time interval;

is the x-th latitude value in the k-th preset time period.

If the k-th preset time interval exists, the similarity S is enabled_i+1，kAnd if the similarity is smaller than the similarity threshold value Tr, dividing ADS-B data in the (i + 1) th preset time interval into X X data blocks by using the X-1 longitude values and the X-1 longitude values extracted in the kth preset time interval.

The key point of this embodiment is that if the ADS-B data in each preset period is divided into X × X data blocks by the above method, X-1 longitude and latitude values in each preset period are different, and the longitude and latitude ranges of each data block in each preset period need to be recorded in the metadata, which consumes a large amount of storage space, and in order to reduce the storage space occupied by such information, this embodiment allows data distribution to have certain flexibility, so as to reuse the existing blocking scheme.

S307, if the k-th preset time interval does not exist in the first i preset time intervals, dividing ADS-B data in the (i + 1) th preset time interval into X X data blocks by using X-1 longitude values and X-1 latitude values extracted in the (i + 1) th preset time interval.

If the similarity is greater than the similarity threshold Tr for any one of the first i periods, extracting X-1 longitude values and X-1 latitude values in the (i + 1) th preset period, and dividing ADS-B data in the (i + 1) th preset period into X data blocks by using the X-1 longitude values and the X-1 latitude values extracted in the (i + 1) th preset period, namely, dividing the X-1 longitude values sequenced in the (i + 1) th preset period into X data blocks

And X-1 latitude value

Dividing ADS-B data in the (i + 1) th preset time interval into X X data blocks for cutting surfaces.

Optionally, the preset time period is 30 minutes, and the time period is a difference between two moments.

Optionally, the number of blocks X is 9, i.e., X is 3.

Optionally, the similarity threshold Tr is 100. Specifically, when the value of Tr is small (Tr is less than or equal to 10), the data set in each time interval is always allocated with a new partitioning scheme, so that the number of partitions is increased continuously, the repeatability of the daily evolution of ADS-B data is utilized when the Tr is increased, the same partitioning scheme can be reused for data sets in the same time period on different dates, particularly when the Tr is increased to 100, the repeated mode is quite obvious, and the total number of partitions is not increased along with the increase of the time, which shows that the existing partitioning scheme can be reused by the new data. . To maximize the reuse efficiency of the chunking scheme, Tr is set to 100.

The data management method provided by this embodiment obtains ADS-B data in an i +1 th preset period, obtains a list of latitude values in the i +1 th preset period and a list of longitude values in the i +1 th preset period, sorts the list of latitude values in the i +1 th preset period, extracts X-1 latitude values at a distribution index such as X-1, sorts the list of longitude values in the i +1 th preset period, extracts X-1 longitude values at a distribution index such as X-1, determines whether a k-th preset period exists in the first i preset periods, determines whether X-1 longitude values and X-1 latitude values extracted in the k-th preset period are similar to X-1 longitude values and X-1 latitude values extracted in the i +1 th preset period, if the k preset time interval exists in the first i preset time intervals, dividing ADS-B data in the (i + 1) th preset time interval into X X data blocks by using X-1 longitude values and X-1 longitude values extracted in the k preset time interval, and if the k preset time interval does not exist in the first i preset time intervals, dividing the ADS-B data in the (i + 1) th preset time interval into X X data blocks by using X-1 longitude values and X-1 latitude values extracted in the (i + 1) th preset time interval. The same blocking scheme can be repeatedly utilized by data in different time periods, and the latitude and longitude ranges of the data blocks are the same, so that the same latitude and longitude range information does not need to be repeatedly stored for each section of data in the metadata, and the storage space required by the metadata is effectively saved.

Fig. 8 is a schematic diagram of a data management apparatus according to an embodiment of the present invention, and as shown in fig. 8, the data management apparatus 80 includes:

an obtaining module 10, configured to obtain a query condition input by a user;

the obtaining module 10 is further configured to obtain a data block corresponding to the query condition from metadata, where the metadata includes a corresponding relationship between the query condition and the data block, and the target data block includes multiple pieces of broadcast-based automatic dependent surveillance (ADS-B) data;

the obtaining module 10 is further configured to obtain target ADS-B data from the target data block.

In one possible design, the obtaining module 10 is further configured to:

the processing module 11 is configured to sort the latitude value list in the ith preset time period, and extract X-1 latitude values at a distribution index such as X-1, where X is an integer greater than or equal to 2; and

In one possible design, the processing module 11 is further configured to:

and compressing ADS-B data in the X data blocks according to columns.

In one possible design, the obtaining module 10 is further configured to:

acquiring ADS-B data in an (i + 1) th preset time period;

the processing module 11 is further configured to:

In one possible design, the processing module 11 is further configured to:

and storing the corresponding relation in the metadata.

In one possible design, the obtaining module 10 is specifically configured to:

The data management device provided by the embodiment of the present invention can implement the above method embodiments, and the implementation principle and technical effect are similar, which are not described herein again.

Fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, and as shown in fig. 8, the electronic device 9 may include:

a memory 902 for storing computer-executable instructions;

the processor 901 is configured to execute the computer-executable instructions stored in the memory to implement the steps performed by the receiving device in the above embodiments. Reference may be made in particular to the description relating to the method embodiments described above.

Alternatively, the memory 902 may be separate or integrated with the processor 901.

When the memory 902 is separately provided, the electronic device 90 further comprises a bus 903 for connecting the memory 902 and the processor 901.

An embodiment of the present invention further provides a computer-readable storage medium, where a computer execution instruction is stored in the computer-readable storage medium, and when a processor executes the computer execution instruction, the data management method is implemented as described above.

In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described device embodiments are merely illustrative, and for example, the division of the modules is only one logical division, and other divisions may be realized in practice, for example, a plurality of modules may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or modules, and may be in an electrical, mechanical or other form.

The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.

In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each module may exist alone physically, or two or more modules are integrated into one unit. The unit formed by the modules can be realized in a hardware form, and can also be realized in a form of hardware and a software functional unit.

The integrated module implemented in the form of a software functional module may be stored in a computer-readable storage medium. The software functional module is stored in a storage medium and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute some steps of the methods according to the embodiments of the present application.

It should be understood that the Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the present invention may be embodied directly in a hardware processor, or in a combination of the hardware and software modules within the processor.

The memory may comprise a high-speed RAM memory, and may further comprise a non-volatile storage NVM, such as at least one disk memory, and may also be a usb disk, a removable hard disk, a read-only memory, a magnetic or optical disk, etc.

The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended ISA (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, the buses in the figures of the present application are not limited to only one bus or one type of bus.

The storage medium may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.

An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be integral to the processor. The processor and the storage medium may reside in an Application Specific Integrated Circuits (ASIC). Of course, the processor and the storage medium may reside as discrete components in an electronic device or host device.

Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. a data management method, is characterized in that, comprises:

Get the query conditions entered by the user;

Obtaining, from the metadata, a target data block corresponding to the query condition, where the metadata includes a correspondence between the query condition and the data block, and the target data block includes a plurality of ADS-B data of broadcast automatic dependent monitoring;

From the target data block, obtain target ADS-B data;

Acquire ADS-B data within the i-th preset time period, each ADS-B data includes aircraft identifier, latitude, longitude, altitude, time, origin, and destination, where i is an integer greater than or equal to 1;

Obtain a list of latitude values in the ith preset time period and a list of longitude values in the ith preset time period, where the list of latitude values in the ith preset time period is the ADS in the ith preset time period The latitude value list of the B data, the longitude value list in the i-th preset time period is the longitude value list of the ADS-B data in the i-th preset time period;

Sorting the list of latitude values in the i-th preset time period, extracting X-1 latitude values at distribution indices such as X-1, where X is an integer greater than or equal to 2; and

Sorting the list of longitude values in the i-th preset time period, extracting X-1 longitude values at the distribution index such as X-1;

According to the X-1 longitude values and X-1 latitude values extracted in the i-th preset time period, the ADS-B data in the i-th preset time period is divided into X*X data blocks.

2. The method according to claim 1, wherein the method further comprises:

The ADS-B data in the X*X data blocks are respectively compressed by column.

3. The method according to claim 1, wherein the method further comprises:

Obtain the ADS-B data within the i+1th preset period;

Obtain a list of latitude values within the i+1th preset period and a list of longitude values within the i+1th preset period, where the list of latitude values within the i+1th preset period is the i+1th preset period. A list of latitude values of the ADS-B data within a preset period, and the list of longitude values within the i+1th preset period is the longitude of the ADS-B data within the i+1th preset period list of values;

Sorting the list of latitude values in the i+1th preset time period, extracting X-1 latitude values at distribution indices such as X-1; and

Sorting the list of longitude values in the i+1th preset time period, and extracting X-1 longitude values at the distribution index such as X-1;

Judging whether there is the kth preset time period in the first i preset time period, the X-1 longitude values and X-1 latitude values extracted in the kth preset time period are the same as the i+1th preset time period. The similarity between the X-1 longitude values and the X-1 latitude values extracted in the time period is less than the similarity threshold, and k is an integer greater than or equal to 1 and less than or equal to i;

If the kth preset time period exists in the first i preset time period, use the X-1 longitude values and X-1 longitude values extracted in the kth preset time period to convert the i+1th longitude value The ADS-B data within the preset period is divided into X*X data blocks.

4. The method according to claim 3, wherein the method further comprises:

If the kth preset time period does not exist in the first i preset time period, use the X-1 longitude values and X-1 latitude values extracted in the i+1th preset time period to The ADS-B data within the +1 preset period is divided into X*X data blocks.

5. The method according to claim 4, wherein the query condition comprises at least one of the following parameters:

Time period, latitude and longitude range, aircraft identifier, origin and destination.

6. The method according to claim 3, wherein the method further comprises:

Establishing the corresponding relationship between the query condition and the data block in advance;

The correspondence is stored in the metadata.

7. The method according to claim 2, wherein the obtaining target ADS-B data from the data block comprises:

According to the query condition, obtain the target data block where the target ADS-B data is located;

According to a user operation, at least one column in the target data block is decompressed, and according to the query condition, the target ADS-B data is obtained from the decompressed at least one column.

The method according to claim 4, wherein the similarity threshold is 100, the number of blocks X*X is 9, and the preset time period is 30 minutes.

9. A data management device, comprising:

The acquisition module is used to acquire the query conditions input by the user;

The obtaining module is further configured to obtain the target data block corresponding to the query condition from the metadata, where the metadata includes the corresponding relationship between the query condition and the data block, and the target data block includes a plurality of broadcast-type data blocks. Automatic correlation monitoring of ADS-B data;

The acquisition module is also used to acquire target ADS-B data from the data block;

The obtaining module is also used for: