[go: up one dir, main page]

CN109670027A - A kind of image query, caching, storing method and system - Google Patents

A kind of image query, caching, storing method and system Download PDF

Info

Publication number
CN109670027A
CN109670027A CN201811607526.0A CN201811607526A CN109670027A CN 109670027 A CN109670027 A CN 109670027A CN 201811607526 A CN201811607526 A CN 201811607526A CN 109670027 A CN109670027 A CN 109670027A
Authority
CN
China
Prior art keywords
service
data
image data
information
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201811607526.0A
Other languages
Chinese (zh)
Other versions
CN109670027B (en
Inventor
唐静芝
陶建林
顾强
黄育华
张立强
刘小栋
徐占海
孙启栓
轩怀亮
何文睿
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
SHANGHAI RURAL COMMERCIAL BANK Co Ltd
Original Assignee
SHANGHAI RURAL COMMERCIAL BANK Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by SHANGHAI RURAL COMMERCIAL BANK Co Ltd filed Critical SHANGHAI RURAL COMMERCIAL BANK Co Ltd
Priority to CN201811607526.0A priority Critical patent/CN109670027B/en
Publication of CN109670027A publication Critical patent/CN109670027A/en
Application granted granted Critical
Publication of CN109670027B publication Critical patent/CN109670027B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Processing Or Creating Images (AREA)

Abstract

The invention discloses a kind of image query, caching, storing method and systems, are related to bank's image and retain field, image storing method, comprising: obtain the corresponding cached configuration strategy of each institution business information;If current time is located at the uplink time section in a cached configuration strategy, the corresponding image data of the corresponding institution business information of the cached configuration strategy in local cache is uploaded to presetting database.The present invention first caches the data that client is uploaded to server, is uploaded to presetting database again during idle time, realizes the read and write abruption of data, optimizes trade system, improves response speed.

Description

A kind of image query, caching, storing method and system
Technical field
The present invention relates to bank's images to retain field more particularly to a kind of image query, caching, storing method and system.
Background technique
In recent years, with the continuous development of banking, business datum is significantly increased, and in process of exchange, is often adjusted Take, store recording, video recording, the unstructured data based on picture carry out authentication etc., traditional retention mode is will be non-structural Change data and structural data (such as: serial number, filename etc.) to be maintained in SAN storage or NAS storage, be read from storage It writes data while carrying out, complete process of exchange.
Compared to structural data, unstructured data is often bigger than structural data, as data on stock constantly increases, Due to the framework limitation of traditional retention mode, the response speed of unstructured data is slower in the data access of high concurrent, and Mode that is existing while being written and read data can further reduced the access of unstructured data to causing centainly to interfere mutually Speed, to reduce the response speed of transaction.
Summary of the invention
The object of the present invention is to provide a kind of image query, caching, storing method and systems, in data high concurrent state Under, it realizes the efficient access of unstructured data, improves transaction response speed, improve client's usage experience.
Technical solution provided by the invention is as follows:
A kind of image storing method, comprising: obtain the corresponding cached configuration strategy of each institution business information;If when current Between be located at a cached configuration strategy in uplink time section when, upload local cache in the cached configuration strategy it is corresponding The corresponding image data of institution business information is to presetting database.
In the above-mentioned technical solutions, big data technology is used, the data access of high concurrent is supported, at the appointed time uploads The image data of caching avoids read-write from interfering, optimizes trade system to presetting database.
Further, the corresponding institution business information of the cached configuration strategy uploaded in local cache is corresponding Image data is to presetting database specifically: the structural data in the image data is uploaded to default first database; By the unstructured data in the image data according to default screening criteria, it is uploaded to the default first database or default Second database.
In the above-mentioned technical solutions, different unstructured datas is stored into different presetting databases, is guaranteed high The transmission speed of speed.
Further, further includes: if current time is more than the uplink time section in a cached configuration strategy, stop uploading The corresponding image data of the corresponding institution business information of the cached configuration strategy.
In the above-mentioned technical solutions, data are only at the appointed time uploaded to presetting database, avoid influencing normal time section Business processing efficiency.
Further, further includes: receive the batch information assembling message that each mechanism is sent;It is assembled according to each batch information Type of service in message judges whether to inquire corresponding buffer service;It, will when inquiring the corresponding buffer service Image data in the corresponding batch information assembling message is cached to local;When not inquiring the corresponding buffer service When, the image data in the corresponding batch information assembling message is uploaded to the presetting database.
In the above-mentioned technical solutions, the specified corresponding image data of type of service is buffered in local, avoids going out simultaneously The read-write phenomenon of existing mass data, realizes read and write abruption.
Further, further includes: the information query message that receiving mechanism is sent;According to the service class in information query message Type judges whether to inquire corresponding buffer service;When not inquiring the corresponding buffer service, from presetting database According to the batch information and/or the corresponding image data of type of service inquiry in information query message;When inquiring corresponding institute When stating buffer service, the batch information and/or the corresponding image data of type of service are inquired from local cache.
In the above-mentioned technical solutions, the corresponding image data of the type of service for opening buffer service is first in local cache Inquiry, can not find out and go in presetting database to inquire again, improve search efficiency.
The present invention also provides a kind of image caching methods, comprising: receives the batch information assembling message that each mechanism is sent;Root According to the type of service in each batch information assembling message, judge whether to inquire corresponding buffer service;When inquiring pair When the buffer service answered, the image data in the corresponding batch information assembling message is cached to local;When not looking into When asking the corresponding buffer service, the image data in the corresponding batch information assembling message is uploaded to described pre- If database.
In the above-mentioned technical solutions, the specified corresponding image data of type of service is buffered in local, avoids going out simultaneously The read-write phenomenon of existing mass data, realizes read and write abruption.
The present invention also provides a kind of image query methods, comprising: the information query message that receiving mechanism is sent;According to information Type of service in query message judges whether to inquire corresponding buffer service;It is taken when not inquiring the corresponding caching When business, from presetting database according in information query message batch information and/or type of service inquire corresponding image number According to;When inquiring the corresponding buffer service, the batch information is inquired from local cache and/or type of service is corresponding Image data.
In the above-mentioned technical solutions, the corresponding image data of buffer service institution business information will be opened first in local cache Middle inquiry can not find out and go in presetting database to inquire again, improve search efficiency.
The present invention also provides a kind of images to retain system, comprising: server cluster;The server cluster includes: strategy Module is obtained, for obtaining the corresponding cached configuration strategy of each institution business information;Data uploading module, if be used for current Between be located at a cached configuration strategy in uplink time section when, upload local cache in the cached configuration strategy it is corresponding The corresponding image data of institution business information is to storing data library.
In the above-mentioned technical solutions, the image data of caching is at the appointed time uploaded to presetting database, realizes read-write point From optimization trade system.
Further, further includes: storing data library;The storing data library includes: the first memory module, described for storing Structural data in image data;And according to default screening criteria, store the structural data in the image data; Second memory module, for storing the structural data in the image data according to screening criteria is preset.
Further, the server cluster further include: message receiving module, the batch information sent for receiving each mechanism Assemble message;Judgment module, for according to the type of service in each batch information assembling message, judging whether to inquire pair The buffer service answered;Cache module, for when inquiring the corresponding buffer service, inciting somebody to action the corresponding batch information group Image data in dress message is cached to local;Data uploading module is further used for that the corresponding caching ought not be inquired When service, the image data in the corresponding batch information assembling message is uploaded to the storing data library.
Compared with prior art, image query of the invention, caching, storing method and retention system beneficial effect are:
The present invention first caches the data that client is uploaded to server, is uploaded to presetting database again during idle time, real The read and write abruption of existing data, optimizes trade system, improves response speed.
Detailed description of the invention
Below by clearly understandable mode, preferred embodiment is described with reference to the drawings, to a kind of image query, caching, Storing method and above-mentioned characteristic, technical characteristic, advantage and its implementation of system are further described.
Fig. 1 is the flow chart of image storing method one embodiment of the present invention;
Fig. 2 is the flow chart of image caching method one embodiment of the present invention;
Fig. 3 is the flow chart of image query method one embodiment of the present invention;
Fig. 4 is the structural schematic diagram that image of the present invention retains system one embodiment;
Fig. 5 is the structural schematic diagram of one embodiment that unstructured data and structural data of the present invention store respectively;
Fig. 6 is the structural representation of one embodiment that unstructured data and structural data of the present invention are stored in Hbase Figure.
Drawing reference numeral explanation:
100. server cluster, 110. strategy acquisition modules, 120. data uploading modules, 130. message receiving modules, 140. service-seeking module, 150. judgment modules, 160. cache modules, 170. information inquiry modules, 200. storing data libraries, 210. first memory modules, 220. second memory modules.
Specific embodiment
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, Detailed description of the invention will be compareed below A specific embodiment of the invention.It should be evident that drawings in the following description are only some embodiments of the invention, for For those of ordinary skill in the art, without creative efforts, it can also be obtained according to these attached drawings other Attached drawing, and obtain other embodiments.
To make simplified form, part related to the present invention is only schematically shown in each figure, they are not represented Its practical structures as product.In addition, there is identical structure or function in some figures so that simplified form is easy to understand Component only symbolically depicts one of those, or has only marked one of those.Herein, "one" is not only indicated " only this ", can also indicate the situation of " more than one ".
The present invention is that the image realized based on big data technology is retained, cached and inquired, and has high-throughput, response speed Fastly, advantage low in cost.
Fig. 1 shows one embodiment of the present of invention, a kind of image storing method, comprising:
S101 server cluster obtains each institution business information (institution business information includes: mechanism number and type of service) Corresponding cached configuration strategy.
S102 (server cluster loops through each cached configuration strategy) is if current time is located in a cached configuration strategy Uplink time section when, upload local cache in the corresponding image data of the corresponding institution business information of cached configuration strategy To presetting database (presetting database is distributed data base);
If S103 current time is more than the uplink time section in a cached configuration strategy, stop uploading the cached configuration The corresponding image data of the corresponding institution business information of strategy.
Specifically, mechanism number refers to the unique identification of mechanism (such as: a bank outlets), for specific mechanism into Row authentication.Type of service refers to the business specifically handled, such as: personal loan, enterprise's monetary transaction, personal monetary transaction Deng.The type of service that each mechanism supports can be identical, can also be different, such as: it self-service can handle hair fastener, inquiry account information etc..
Image data includes structural data and unstructured data, structural data include: mechanism number, type of service, Batch information, exchange hour etc., unstructured data include: the picture file of the secondary transaction, video file etc..
There may be differences for each mechanism number cached configuration strategy corresponding with type of service, such as: business is busier Mechanism cached configuration strategy are as follows: belong to all types of service of this mechanism image data be after local cache 3 days, evening Upper 10 points just upload to 8 points of morning;The cached configuration strategy of the more idle mechanism of business (substantially without client after at 3 points in afternoon) Are as follows: belong to the image data of all types of service of this mechanism after local cache 7 days, 8 points to morning of at 3 points in afternoon carries out It passes.
Server cluster loops through all cached configuration strategies, is confirmed whether to need to upload shadow according to current time As data to presetting database;If current time, not in uplink time section, no matter whether image data has uploaded, all no longer It uploads, in order to avoid influence the normal operation respectively traded in mechanism.
Such as: mechanism A is a bank outlets, and bank's working time is 9:00-16:30 points, Monday to Saturday, at this In period, the client of mechanism A is relatively more, and query information, the frequency of upload information to server cluster are higher, if at this It waits server cluster and the mechanism A image data uploaded is directly uploaded to presetting database, read and write while carrying out, it can be in certain journey The response speed for transferring, storing image data in sequence in influence process of exchange, causes mechanism A poor to the usage experience of user. The corresponding cached configuration strategy of the mechanism number of mechanism A can be pre- to be uploaded to 7 points of every night to 8 points of morning in server cluster If database, the image data that other times mechanism A is uploaded first is buffered in the local of server cluster.Certainly, mechanism A's is every A type of service can configure different cached configuration strategies, meet different use demands.
Therefore, the cached configuration strategy and institution business according to different use requirements, in actual conditions server cluster Information is corresponding, and a server cluster corresponds to several mechanisms, and there are several types of service in a mechanism, can there is multiple cachings Configuration strategy.
Server cluster loops through each cached configuration strategy, if current time is in the upload of a cached configuration strategy When time interval, then the corresponding image data of the corresponding institution business information of the cached configuration strategy is uploaded;If at current time At the uplink time section of multiple cached configuration strategies, then multiple threads is called to upload the corresponding mechanism of each cached configuration strategy The corresponding image data of business information.If current time is located at uplink time section, but without image data need to upload when, Then thread suspend mode, when the image data and current time of upload in need are located at the upload in a certain cached configuration strategy Between section.
Preferably, upload local cache in the corresponding image data of the corresponding institution business information of cached configuration strategy extremely Presetting database specifically: obtain the batch information of the corresponding institution business information of the cached configuration strategy, and the batch is believed It ceases corresponding image data and is uploaded to presetting database.
Specifically, batch information includes: batch serial number, it is the unique information mark of every data;Name, gender etc..
Optionally, it can be obtained in batches when obtaining batch information, when upload can be realized batch and upload.Such as: it is primary to obtain 10 batch informations, while uploading this corresponding image data of 10 batch informations.
It should be noted that the unstructured data in a image data can include multiple pictures, video file simultaneously. Such as: user wants to handle out card, and this time the corresponding unstructured data of batch information of transaction has the positive and negative figure of identity card Piece, the video file etc. of the face recognition process of user.
The corresponding image data of the corresponding institution business information of cached configuration strategy in local cache is uploaded to present count Control strategy is used during according to library;Control strategy includes any of the following or a variety of: flow control, data compression, excellent Select grade control, breakpoint transmission.
Flow control refers to, carries out flow control when uploading to the image data of every kind of type of service, by server according to It actually uses situation and carries out dynamic or static adjustment.
Data compression refers to, carries out compression of images when migrating to presetting database to one or several file types, by Server configuration.
Two priority classes refer to, to type of service present on server, carry out arrangement upload according to priority and (refer to simultaneously When needing to upload), priority is configured by engineer.
Breakpoint transmission refers to, when transmitting more image files, transmission is encountered when unsuccessfully needing to transmit again, only to preceding primary The file for uploading failure is transmitted again, improves efficiency of transmission.
The equipment of control strategy improves the efficiency that server cluster uploads data, and the higher data of priority can be excellent It first migrates, protects the safety of significant data.
Preferably, upload local cache in the corresponding image data of the corresponding institution business information of cached configuration strategy extremely Presetting database specifically:
Structural data in image data is uploaded to default first database;
By the unstructured data in image data according to default screening criteria, it is uploaded to default first database or default Second database.
Default screening criteria includes any of the following or a variety of: the configuration such as type of service, image modality, format, size Parameter.
Specifically, default first database is HBASE, Hive distributed column database (peculiar number of big data technology According to memory module, the data access of high concurrent is supported);Default second database is HDFS (Hadoop distributed file system), It can provide the data access of high-throughput, the application being very suitable on large-scale dataset.Image file is this bigger File data store to HDFS, one comes that file transfer speed is higher, and two carry out user in client query, transfers speed more Fastly, response speed faster, makes user have good usage experience.
Structural data is smaller, is directly stored in HBASE, does not use HDFS structured data etc. lesser Data are because the minimum unit of its memory block is larger, if storage small documents, can have a large amount of redundancy and waste, cause to imitate Rate is low;And unstructured data then needs to carry out selection storage according to default screening criteria, improves the use effect of each database Rate.
The example of each default screening criteria in actual use is as follows:
Type of service: certain types of service are typically all video file (such as: recording the movement done of user), very greatly, It can be directly confirmed whether to be stored in HDFS according to type of service.
Image modality: the image modality of certain unstructured datas is video, then it can be allowed to be stored in HDFS, if figure Piece is then judged according to subsequent format, size again.
Format: certain formats are high definition pictures, and certain formats are video files, and the difference of format represents these data It is of different sizes, therefore, it can be screened and be stored to corresponding presetting database according to format.
Size: as soon as dividing a boundary, greater than storing to default second database for this boundary, what is be less than or equal to is just deposited It stores up to default first database.
Default screening criteria is according to the actual situation by engineer's self-setting.
As shown in figure 5, be to store structural data to Hbase, the case where unstructured data is stored to HDFS;Such as It is the case where all storage structural data and unstructured data to Hbase shown in Fig. 6.
Optionally, after image data is uploaded to presetting database, it is corresponding in local cache to modify the image data Buffer status.
Specifically, being managed using data cached list to the data in local cache, their buffer status is inquired, Include: to have cached, had moved (or upload) etc., adjusts according to demand.
In the present embodiment, image data is stored using big data technology, supports the data access of high concurrent, it is real The efficient storage of unstructured data is showed, and has also laid the foundation for quickly reading.In addition, server cluster is matched according to caching It sets strategy and uploads the problem of being buffered in local image data, avoiding read-write interference again during idle time, compared to existing tradition Under retention mode, there is the phenomenon that largely reading while write in image class business, right under the big data distributed system of the present embodiment Image transaction is written and read separation, optimizes process of exchange, improves response speed.
Fig. 2 shows one embodiment of image caching method of the present invention, comprising:
S201 server cluster receives the batch information assembling message that each mechanism is sent;
S203 server cluster assembles the type of service in message according to each batch information, judges whether to inquire respectively pair The buffer service (judging whether to enable buffer service) answered;
For S204 when inquiring corresponding buffer service, corresponding batch information is assembled the shadow in message by server cluster As data buffer storage to local (being provided with CACHE DIRECTORY in buffer service, will be in image data caching so far CACHE DIRECTORY);
For S205 when not inquiring corresponding buffer service, server cluster assembles corresponding batch information in message Image data is uploaded to presetting database.
Specifically, engineer on server cluster according to actual needs, configure corresponding caching for each type of service Service (unlatching/closing, if opening, which CACHE DIRECTORY is) certainly can also be corresponding slow according to each institution business information configuration The service of depositing.
It includes: the keys such as batch information, type of service, mechanism number, file size, image file that batch information, which assembles message, Business information.
Optionally, between S201 and S203 further include: S201 server cluster is according to each institution business information (i.e. service class Type and mechanism number) inquire the state of corresponding mechanism respectively, if do not inquire or the state of mechanism be abnormality, to the machine Structure returns to exception information, and abandons corresponding batch information assembling message;When state is normal condition, S203 is executed.
Whether the state that this type of service under corresponding mechanism is inquired by mechanism number is normal, that is, judges whether the mechanism has execution The permission of this type of service can just confirm buffer service if normal, with the operation for executing caching or uploading.
If inquiring according to mechanism number and/or type of service and enabling buffer service, which is assembled into message In image data cache to local corresponding CACHE DIRECTORY, to subsequent arrival uplink time section, then be uploaded to default clothes Business device.
If inquiring not enabled buffer service according to mechanism number and/or type of service, which is assembled into message In image data directly upload presetting database.
Preferably, the structural data in image data is uploaded to default first database;It will be non-in image data Structural data is uploaded to default first database or default second database according to default screening criteria.
Default screening criteria includes any of the following or a variety of: the configuration such as type of service, image modality, format, size Parameter.Identical in practice as above-mentioned image storing method, details are not described herein.
In the present embodiment, server cluster is after the image data for receiving mechanism transmission, according to the configuration of buffer service It is first buffered in local, is uploaded to presetting database again when subsequent idle, separation is written and read to image class business transaction, is optimized Trade system improves response speed.
Fig. 3 shows one embodiment of image query method of the present invention, comprising:
The information query message that S301 server cluster receiving mechanism is sent.
S303 server cluster judges whether to inquire corresponding caching clothes according to the type of service in information query message Business (judges whether to enable buffer service);Certainly, can also be according to the corresponding buffer service of each institution business information configuration, then pass through Institution business information judges whether there is corresponding buffer service.
S304 when not inquiring corresponding buffer service, inquire from presetting database according to information and report by server cluster Batch information and/or type of service in text inquire corresponding image data.
S305 when inquiring corresponding buffer service, server cluster inquired from local cache batch information and/or The corresponding image data of type of service.
S306 is when being not present batch information and/or the corresponding image data of type of service in local cache, from preset data Batch information and/or the corresponding image data of type of service are inquired in library.
Specifically, information query message includes: batch information, mechanism number and type of service etc..
Optionally, between S301 and S302 further include: S302 is according to institution business information (the i.e. machine in information query message Structure number and type of service) the corresponding mechanism of inquiry state, if not inquiring or the state of mechanism is abnormality, to the machine Structure returns to exception information, and abandons information query message;When state is normal condition, S303 is executed.
Specifically, whether the state that this type of service under corresponding mechanism is inquired by mechanism number is normal, i.e., whether decision mechanism There is the permission for executing this type of service that can just confirm buffer service if normal, to execute the operation of caching or inquiry.
If inquiring according to mechanism number and/or type of service and enabling buffer service, illustrate can exist in local cache Certain image data can first look into local cache, call relevant structuring or unstructured data, if not having, then look into default Database;If according to mechanism number and/or type of service (in other embodiments, also only on the basis of batch serial number and mechanism number Inquiry) inquire not enabled buffer service, then explanation can only be inquired from presetting database.
It is inquired using type of service and/or batch information, can be realized the data query across mechanism.
Optionally, when local cache is inquired, the state of each data can be inquired from data cached list, it is determined whether also It is stored in local cache, improves search efficiency.
In the present embodiment, if needing to call the image data of storage in process of exchange and it is stored in local, inquire Speed improves the response speed of transaction far faster than from preset data library inquiry on certain procedures.And whole system uses Big data technology supports the data access of high concurrent, has faster response speed and treatment effeciency in process of exchange, significantly Improve the usage experience of client.
In the embodiment of another image storing method of the invention, comprising:
Server cluster receives the batch information assembling message that each mechanism is sent;
Server cluster inquires corresponding client according to each institution business information (i.e. type of service and mechanism number) respectively State return to exception information if not inquiring or the state of mechanism is abnormality to the mechanism, and abandon batch information Assemble message;
When state is normal condition, server cluster assembles the type of service in message according to corresponding batch information, Judge whether to inquire corresponding buffer service (judging whether to enable buffer service);
When inquiring corresponding buffer service, corresponding batch information is assembled the image number in message by server cluster According to caching to local (being provided with CACHE DIRECTORY in buffer service, will be in image data caching so far CACHE DIRECTORY);
When not inquiring corresponding buffer service, corresponding batch information is assembled the image in message by server cluster Data are uploaded to presetting database.
Server cluster receives the information query message that (each) mechanism is sent;
Server cluster inquires the shape of corresponding mechanism according to the institution business information in (each) information query message respectively State returns to exception information to the mechanism, and abandons corresponding information and look into if not inquiring or the state of mechanism is abnormality Ask message;
When state is normal condition, server cluster is sentenced according to the type of service in each corresponding information query message It is disconnected whether to inquire corresponding buffer service (judging whether to enable buffer service);
When not inquiring corresponding buffer service, server cluster from presetting database according in information query message Batch information and/or type of service inquire corresponding image data;
When inquiring corresponding buffer service, server cluster inquires batch information and/or business from local cache The corresponding image data of type;
When there is no batch information and/or the corresponding image datas of type of service in local cache, from presetting database Inquire batch information and/or the corresponding image data of type of service.
It is corresponding that server cluster obtains each institution business information (institution business information includes: mechanism number and type of service) Cached configuration strategy.
Server cluster loops through each cached configuration strategy, if current time is located at the upload in a cached configuration strategy When time interval, the corresponding image data of the corresponding institution business information of cached configuration strategy in local cache is uploaded to default Database;
If current time is more than the uplink time section in a cached configuration strategy, server cluster stops upload, and this is slow Deposit the corresponding image data of the corresponding institution business information of configuration strategy.
Preferably, upload local cache in the corresponding image data of the corresponding institution business information of cached configuration strategy extremely Presetting database specifically: obtain the batch information of the corresponding institution business information of the cached configuration strategy, and the batch is believed It ceases corresponding image data and is uploaded to presetting database.
Optionally, upload local cache in the corresponding image data of the corresponding institution business information of cached configuration strategy extremely Control strategy is used during presetting database;Control strategy includes any of the following or a variety of: flow control, data pressure Contracting, priority control, breakpoint transmission.
Preferably, upload local cache in the corresponding image data of the corresponding institution business information of cached configuration strategy extremely Presetting database specifically: the structural data in image data is uploaded to default first database;It will be in image data Unstructured data is uploaded to default first database or default second database according to default screening criteria.Default screening mark Standard includes any of the following or a variety of: the configuration parameters such as type of service, image modality, format, size.
Optionally, after image data is uploaded to presetting database, it is corresponding in local cache to modify the image data Buffer status.
The present embodiment its implementation process of part identical with above-mentioned each method embodiment is identical, specifically refers to above-mentioned each reality Example is applied, details are not described herein.
The present embodiment first caches the data that mechanism is uploaded to server cluster, under big data distributed system in sky Idle is uploaded to presetting database again, realizes the read and write abruption of data, optimizes trade system;It is each machine according to different demands Structure and its type of service configure different cached configuration strategies, unlatching/closing buffer service, lay base for improve data transfer Plinth;It when inquiry, if opening buffer service, is first inquired in local cache, avoids the write-in of high-volume data, inquires mutual shadow It rings, improves response speed.
Fig. 4 shows the embodiment that one image of the present invention retains system, comprising: server cluster 100;
Server cluster 100 includes:
Strategy obtains module 110, corresponding for obtaining each institution business information (including: mechanism number and type of service) Cached configuration strategy;
Data uploading module 120, for (looping through each cached configuration strategy) if current time is located at a cached configuration When uplink time section in strategy, the corresponding shadow of the corresponding institution business information of cached configuration strategy in local cache is uploaded As (presetting database described in i.e. in above method embodiment, presetting database is distributed data to data to storing data library Library);And if stop uploading the cached configuration plan when current time is more than the uplink time section in a cached configuration strategy The corresponding image data of slightly corresponding institution business information.
Specifically, mechanism number refers to the unique identification of mechanism (such as: a bank outlets), for specific mechanism into Row authentication.Type of service refers to the business specifically handled, such as: personal loan, enterprise's monetary transaction, personal monetary transaction Deng.The type of service that each mechanism supports can be identical, can also be different, such as: it self-service can handle hair fastener, inquiry account information etc..
Image data includes structural data and unstructured data, structural data include: mechanism number, type of service, Batch information, exchange hour etc., unstructured data include: the picture file of the secondary transaction, video file etc..
There may be difference, server clusters to loop through for each mechanism number cached configuration strategy corresponding with type of service All cached configuration strategies are confirmed whether to need to upload according to current time image data to presetting database;If current Time, no matter whether image data uploaded not in uplink time section, all no longer uploaded, in order to avoid influence respectively to trade in mechanism Normal use experience.
Specific example can be found in the example in image storing method, and details are not described herein.
According to different use requirements, the cached configuration strategy and institution business information pair in actual conditions server cluster It answers, a server cluster corresponds to several mechanisms, and there are several types of service in a mechanism, can there is multiple cached configuration plans Slightly.
Server cluster loops through each cached configuration strategy, if current time is in the upload of a cached configuration strategy When time interval, then the corresponding image data of the corresponding institution business information of the cached configuration strategy is uploaded;If at current time At the uplink time section of multiple cached configuration strategies, then multiple threads is called to upload the corresponding mechanism of each cached configuration strategy The corresponding image data of business information.When being located at uplink time section immediately, but when needing to upload without image data, then thread Suspend mode, until the image data and current time of upload in need are located at the uplink time section in a certain cached configuration strategy.
Preferably, data uploading module 120 uploads the corresponding institution business information of cached configuration strategy in local cache Corresponding image data is to storing data library specifically:
Data uploading module 120 obtains the batch information of the corresponding institution business information of the cached configuration strategy, and should The corresponding image data of batch information is uploaded to storing data library.
Specifically, batch information includes: batch serial number, it is the unique information mark of every data;Name, gender etc..
Optionally, it can be obtained in batches when obtaining batch information, when upload can be realized batch and upload.Such as: it is primary to obtain 10 batch informations, while uploading this corresponding image data of 10 batch informations.
It should be noted that the unstructured data in a image data can include multiple pictures, video file simultaneously. Such as: user wants to handle out card, and this time the corresponding unstructured data of batch information of transaction has the positive and negative figure of identity card Piece, the video file etc. of the face recognition process of user.
The corresponding image data of the corresponding institution business information of cached configuration strategy in local cache is uploaded to present count Control strategy is used during according to library;Control strategy includes any of the following or a variety of: flow control, data compression, excellent Select grade control, breakpoint transmission.It is specific to explain that details are not described herein referring to image storing method embodiment.
The equipment of control strategy improves the efficiency that server uploads data, and the higher data of priority can be moved preferentially It moves, protects the safety of significant data.
Preferably, image retains system, further includes: storing data library 200.Storing data library includes:
First memory module 210, for storing the structural data in image data;And according to default screening criteria, Store the structural data in image data;
Second memory module 220, for storing the structural data in image data according to screening criteria is preset.
Default screening criteria includes any of the following or a variety of: the configuration such as type of service, image modality, format, size Parameter.
Specifically, default first database is HBASE, Hive distributed column database (peculiar number of big data technology According to memory module, the data access of high concurrent is supported);Default second database is HDFS, it can provide the data of high-throughput Access, the application being very suitable on large-scale dataset.This bigger file data of image file is stored to HDFS, one It is higher to carry out file transfer speed, two carry out user in client query, transfer speed faster, and response speed faster, there is user Good usage experience.
Structural data is smaller, is directly stored in HBASE, does not use HDFS structured data etc. lesser Data are because the minimum unit of its memory block is larger, if storage small documents, can have a large amount of redundancy and waste, cause to imitate Rate is low;And unstructured data then needs to carry out selection storage according to default screening criteria, improves the use effect of each database Rate presets screening criteria according to the actual situation by engineer's self-setting.The example of each default screening criteria in actual use Image storing method embodiment is referred to, details are not described herein.
Optionally, after image data is uploaded to storing data library, it is corresponding in local cache to modify the image data State.
Specifically, being managed using data cached list to the data in local cache, their buffer status is inquired, Include: to have cached, had moved (or upload) etc., adjusts according to demand.
Optionally, server cluster 100 further include:
Message receiving module 130 assembles message for receiving the batch information that each mechanism is sent;
Service-seeking module 140, for assembling the institution business information enquiry machine structure in message according to each batch information State, if do not inquire or the state of mechanism be abnormality, to mechanism return exception information, and abandon batch information assembling Message;
Judgment module 150, for assembling the type of service in message according to batch information when state is normal condition, Judge whether to inquire corresponding buffer service (judging whether to enable buffer service);
Cache module 160, for when inquiring corresponding buffer service, corresponding batch information to be assembled in message Image data is cached to local (being provided with CACHE DIRECTORY in buffer service, will be in image data caching so far CACHE DIRECTORY);
Data uploading module 120 is further used for when not inquiring corresponding buffer service, by corresponding batch information Image data in assembling message is uploaded to storing data library.
Specifically, engineer is on the server according to actual needs, corresponding buffer service is configured for each type of service (unlatching/closing, if opening, which CACHE DIRECTORY is) can also take certainly according to the corresponding caching of each institution business information configuration Business.
It includes: the keys such as batch information, type of service, mechanism number, file size, image file that batch information, which assembles message, Business information.
Whether the state that this type of service under corresponding mechanism is inquired by mechanism number is normal, that is, judges whether the mechanism has execution The permission of this type of service can just confirm buffer service if normal, with the operation for executing caching or uploading.
If inquiring according to mechanism number and/or type of service and enabling buffer service, which is assembled into message In image data cache to local corresponding CACHE DIRECTORY, to subsequent arrival uplink time section, then be uploaded to default clothes Business device.
If inquiring not enabled buffer service according to mechanism number and/or type of service, which is assembled into message In image data directly upload presetting database.
Optionally, message receiving module 130 is further used for receiving the information query message that each mechanism is sent;
Service-seeking module 140 is inquired pair respectively for assembling the institution business information in message according to each batch information The state for the mechanism answered returns to exception information to the mechanism, and abandons if not inquiring or the state of mechanism is abnormality Batch information assembles message;
Judgment module 150, for when state be normal condition when, according in information query message type of service and/or Mechanism number judges whether to inquire corresponding buffer service (judging whether to enable buffer service).
Server cluster further include: information inquiry module 170, for when not inquiring corresponding buffer service, from depositing It stores up in database according to the batch information and/or the corresponding image data of type of service inquiry in information query message;And when When inquiring corresponding buffer service, batch information and/or the corresponding image data of type of service are inquired from local cache;With And when batch information and/or the corresponding image data of type of service are not present in local cache, inquiry is criticized from presetting database Secondary information and/or the corresponding image data of type of service.
Specifically, information query message includes: batch information, mechanism number and type of service etc..
Specifically, whether the state that this type of service under corresponding mechanism is inquired by mechanism number is normal, i.e., whether decision mechanism There is the permission for executing this type of service that can just confirm buffer service if normal, with the operation for executing caching or uploading.
If inquiring according to mechanism number and/or type of service and enabling buffer service, illustrate can exist in local cache Certain image data can first look into local cache, call relevant structuring or unstructured data, if not having, then look into default Database;If according to mechanism number and/or type of service (in other embodiments, also only on the basis of batch serial number and mechanism number Inquiry) inquire not enabled buffer service, then explanation can only be inquired from presetting database.
It is inquired using type of service and/or batch information, can be realized the data query across mechanism.
Optionally, when local cache is inquired, the state of each data can be inquired from data cached list, it is determined whether also It is stored in local cache, improves search efficiency.
The present embodiment first caches the data that each mechanism is uploaded to server cluster under big data distributed system, Presetting database is uploaded to when idle again, realizes the read and write abruption of data, optimizes trade system;It is each according to different demands Mechanism and its type of service configure different cached configuration strategies, unlatching/closing buffer service, lay base for improve data transfer Plinth;It when inquiry, if opening buffer service, is first inquired in local cache, avoids the write-in of high-volume data, inquires mutual shadow It rings, improves response speed.
Above-mentioned image retains system, and specific implementation is to be realized using big data platform to image number in actual use According to unstructured data carry out efficient storage, extract.Big data image platform includes three-tier architecture, and bottom is data storage Layer, can be chosen automatically according to different parameters, different business different storage models (store to HDFS or Hbase, it is existing Be to be stored only with SAN, all unstructured datas and structural data are stored in SAN storage), and to rope Argument evidence and image file carry out unified storage and management, and unified supporting is provided for upper layer application.
Middle layer is application service layer, comprising configuration management module, cache module, api interface module etc., wherein caching mould Block can be realized according to configuration is directed to different business, different time business datum read and write abruption under big data.
Top layer is service access layer, by being uniformly accessed into console module management, each peripheral applications according to respective business needs, Technical conditions can realize the different connection modes such as control load, the page are embedded in and API is direct-connected respectively, complete scanning record, delete The business functions such as remove, update, inquiring, showing, annotating.
Big data platform of the invention has carried out data and service assembly, changes original image data access module, Simplify out-out business system access process, difficulty.Under new architecture storage model uniformly transfer to data storage layer creation, maintenance and Management, using general image store model, as the structural datas such as image batch serial number, type of service, exchange hour are united One storage to HBASE, HIVE distributed column database (sql like language system), as this kind of unstructured data of image file according to According to the performance test results, preservation can be automatically selected according to configuration parameters such as type of service, image modality, format, sizes extremely In HBASE or HDFS (Hadoop distributed file system needs 2 layers of access under this mode, is suitble to big file), and externally provide Transparent service.
By taking intelligent automatic teller machine (as a client under a mechanism) related service as an example, the public process of transaction It is as follows:
1, client applies
Client is inserted into debit card or financial services card, selects the business for needing to handle after verifying password, chooses and handle content.
2, authentication
Client provides identity card original part, by system automatic network verification and retains image, client signs electronic signature and carries out industry Business confirmation.
3, business is audited
Certificate image, verification image, live full face and the type of service handled are pushed to lobby manager's hand by system It holds on PAD, is audited and confirmed by lobby manager, submit transaction after confirming successfully.(authorization can be configured according to business risk degree Mode: without authorization, PAD authorization, on-the-spot audit authorization).
4, result returns
System returns the result to intelligent automatic teller machine after trading successfully, prints corresponding receipt or strip.
5, follow-up business
If client need it is subsequent handle other front-end business, the function that may be selected to call out the numbers is waited in line.
Wherein authentication link is related to image upload transaction, and audit link is related to image query transaction, and detailed process is such as Under:
Authentication: in transacting business, identity card is read client automatically, carries out online checking, retains certificate image, and Floor client's full face carries out recognition of face by intelligent automatic teller machine and is compared, by comparing result, the service class handled The image datas such as type and certificate image, verification image, live full face are uploaded to image and retain system.
Audit business: intelligent automatic teller machine push authentication relevant information to lobby manager holds PAD (i.e. another client End) on, PAD retains system from image and obtains coherent video picture, executes different auditing standards in conjunction with system identification degree index, such as Fruit system identification degree can be carried out on the spot audit fingerprint identification by lobby manager lower than certain proportion, can be with higher than certain proportion Non-at-scene audit is carried out on PAD.
Meanwhile the scene photograph that client shoots when only needing to first time transacting business is retained, for subsequent transacting business Shi Jinhang authentication provides reference, and follow-up checks only need to inquire existing image retained data.
It should be noted that above-described embodiment can be freely combined as needed.The above is only of the invention preferred Embodiment, it is noted that for those skilled in the art, in the premise for not departing from the principle of the invention Under, several improvements and modifications can also be made, these modifications and embellishments should also be considered as the scope of protection of the present invention.

Claims (10)

1. a kind of image storing method characterized by comprising
Obtain the corresponding cached configuration strategy of each institution business information;
If current time is located at the uplink time section in a cached configuration strategy, the caching uploaded in local cache is matched The corresponding image data of the corresponding institution business information of strategy is set to presetting database.
2. image storing method as described in claim 1, which is characterized in that the caching uploaded in local cache The corresponding image data of the corresponding institution business information of configuration strategy is to presetting database specifically:
Structural data in the image data is uploaded to default first database;
By the unstructured data in the image data according to default screening criteria, be uploaded to the default first database or Default second database.
3. image storing method as described in claim 1, which is characterized in that further include:
If current time is more than the uplink time section in a cached configuration strategy, it is corresponding that stopping uploads the cached configuration strategy The corresponding image data of institution business information.
4. image storing method as described in claim 1, which is characterized in that further include:
Receive the batch information assembling message that each mechanism is sent;
According to the type of service in each batch information assembling message, judge whether to inquire corresponding buffer service;
When inquiring the corresponding buffer service, the image data in the corresponding batch information assembling message is cached To local;
It, will be on the image data in the corresponding batch information assembling message when not inquiring the corresponding buffer service Reach the presetting database.
5. image storing method as described in claim 1, which is characterized in that further include:
The information query message that receiving mechanism is sent;
According to the type of service in information query message, judge whether to inquire corresponding buffer service;
When not inquiring the corresponding buffer service, from presetting database according to the batch information in information query message And/or type of service inquires corresponding image data;
When inquiring the corresponding buffer service, the batch information and/or type of service pair are inquired from local cache The image data answered.
6. a kind of image caching method characterized by comprising
Receive the batch information assembling message that each mechanism is sent;
According to the type of service in each batch information assembling message, judge whether to inquire corresponding buffer service;
When inquiring the corresponding buffer service, the image data in the corresponding batch information assembling message is cached To local;
It, will be on the image data in the corresponding batch information assembling message when not inquiring the corresponding buffer service Reach the presetting database.
7. a kind of image query method characterized by comprising
The information query message that receiving mechanism is sent;
According to the type of service in information query message, judge whether to inquire corresponding buffer service;
When not inquiring the corresponding buffer service, from presetting database according to the batch information in information query message And/or type of service inquires corresponding image data;
When inquiring the corresponding buffer service, the batch information and/or type of service pair are inquired from local cache The image data answered.
8. a kind of image retains system characterized by comprising server cluster;
The server cluster includes:
Strategy obtains module, for obtaining the corresponding cached configuration strategy of each institution business information;
If data uploading module uploads local be located at the uplink time section in a cached configuration strategy for current time The corresponding image data of the corresponding institution business information of the cached configuration strategy in caching is to storing data library.
9. image as claimed in claim 8 retains system, which is characterized in that further include: storing data library;
The storing data library includes:
First memory module, for storing the structural data in the image data;And it according to default screening criteria, deposits Store up the structural data in the image data;
Second memory module, for storing the structural data in the image data according to screening criteria is preset.
10. image as claimed in claim 8 retains system, which is characterized in that the server cluster further include:
Message receiving module assembles message for receiving the batch information that each mechanism is sent;
Judgment module, for judging whether to inquire corresponding according to the type of service in each batch information assembling message Buffer service;
Cache module, for when inquiring the corresponding buffer service, the corresponding batch information to be assembled in message Image data cache to local;
Data uploading module is further used for when not inquiring the corresponding buffer service, will the corresponding batch letter Image data in breath assembling message is uploaded to the storing data library.
CN201811607526.0A 2018-12-27 2018-12-27 Image query, cache and retention method and system Active CN109670027B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811607526.0A CN109670027B (en) 2018-12-27 2018-12-27 Image query, cache and retention method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811607526.0A CN109670027B (en) 2018-12-27 2018-12-27 Image query, cache and retention method and system

Publications (2)

Publication Number Publication Date
CN109670027A true CN109670027A (en) 2019-04-23
CN109670027B CN109670027B (en) 2021-05-11

Family

ID=66147330

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811607526.0A Active CN109670027B (en) 2018-12-27 2018-12-27 Image query, cache and retention method and system

Country Status (1)

Country Link
CN (1) CN109670027B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113726903A (en) * 2021-09-03 2021-11-30 中国银行股份有限公司 Data uploading method and device
CN116128636A (en) * 2023-03-07 2023-05-16 苏银凯基消费金融有限公司 Efficient management system and method for business image data

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102270161A (en) * 2011-06-09 2011-12-07 华中科技大学 Methods for storing, reading and recovering erasure code-based multistage fault-tolerant data
CN104850509A (en) * 2015-04-27 2015-08-19 交通银行股份有限公司 Method and system for operating bank business data memory cache
CN106095796A (en) * 2016-05-30 2016-11-09 中国邮政储蓄银行股份有限公司 Distributed data storage method, Apparatus and system
CN108053313A (en) * 2018-01-02 2018-05-18 中国工商银行股份有限公司 Cross-border data processing method of opening an account, apparatus and system
CN108319542A (en) * 2017-01-17 2018-07-24 百度在线网络技术(北京)有限公司 Information processing method, apparatus and system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102270161A (en) * 2011-06-09 2011-12-07 华中科技大学 Methods for storing, reading and recovering erasure code-based multistage fault-tolerant data
CN104850509A (en) * 2015-04-27 2015-08-19 交通银行股份有限公司 Method and system for operating bank business data memory cache
CN106095796A (en) * 2016-05-30 2016-11-09 中国邮政储蓄银行股份有限公司 Distributed data storage method, Apparatus and system
CN108319542A (en) * 2017-01-17 2018-07-24 百度在线网络技术(北京)有限公司 Information processing method, apparatus and system
CN108053313A (en) * 2018-01-02 2018-05-18 中国工商银行股份有限公司 Cross-border data processing method of opening an account, apparatus and system

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113726903A (en) * 2021-09-03 2021-11-30 中国银行股份有限公司 Data uploading method and device
CN113726903B (en) * 2021-09-03 2022-09-20 中国银行股份有限公司 Data uploading method and device
CN116128636A (en) * 2023-03-07 2023-05-16 苏银凯基消费金融有限公司 Efficient management system and method for business image data
CN116128636B (en) * 2023-03-07 2024-11-29 苏银凯基消费金融有限公司 Efficient management system and method for business image data

Also Published As

Publication number Publication date
CN109670027B (en) 2021-05-11

Similar Documents

Publication Publication Date Title
US11397744B2 (en) Systems and methods for data storage and processing
KR102026225B1 (en) Apparatus for managing data using block chain and method thereof
CN110032571A (en) Business flow processing method, apparatus, storage medium and calculating equipment
CN109074387A (en) Versioned Hierarchical Data Structures in Distributed Datastores
US11783074B2 (en) Controlling access to secure information resources using rotational datasets and dynamically configurable data containers
US20110196828A1 (en) Method and System for Dynamically Replicating Data Within A Distributed Storage System
CN109582722A (en) Public security resource data service system
KR20240020168A (en) Appratus and method for ESG management that facilitates response to internal and external ESG needs
CN107169862B (en) Automatic tracking system for deposit of bad bank customer
US12069057B2 (en) Controlling access to secure information resources using rotational datasets and dynamically configurable data containers
CN109639598A (en) Request processing method, server, storage medium and device based on micro services
CN109948373A (en) A kind of multiparty service data interactive method
US12056153B2 (en) Replicating large statements with low latency
CN110442561A (en) Distributed file storage system and its storage method based on block chain
KR102475435B1 (en) Apparatus for managing data using block chain and method thereof
CN109670027A (en) A kind of image query, caching, storing method and system
CN112883091A (en) Factor data acquisition method and device, computer equipment and storage medium
US20200134608A1 (en) Systems and Methods for Charitable Giving Using Blockchain Cryptocurrency
CN108268468A (en) The analysis method and system of a kind of big data
CN113111125A (en) Business evidence storage method based on block chain
US7734785B2 (en) On demand message based financial network integration middleware
CN119583657A (en) Network service method, device and medium
CN109347896A (en) A kind of information processing method, equipment and computer readable storage medium
CN113221177A (en) Data access method, device and system in distributed system
CN111209414A (en) Method for realizing cold-hot separated storage of data based on image data calling service scene

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant