CN110612697B - Method and system for efficient information retrieval of data storage layer indexes - Google Patents

Abstract

A secure data system facilitates efficient searching of a wide data storage layer (e.g., blockchain). The system may create an index for a predefined type of data element recorded in the blockchain. The system may generate the index by invoking executable instructions stored in executable data elements (e.g., smart contracts) that are recorded in the blockchain and associated with a predefined type of data element. The index facilitates quick querying of predefined types of data elements in the blockchain.

Description

Method and system for data storage layer indexing for efficient information retrieval

Technical field

This disclosure relates to secure data systems and to more efficiently locating data elements in a secure data storage layer, such as a blockchain.

Background technique

Rapid advances in electronics and communications technology, driven by overwhelming customer demand, have led to the widespread adoption of secure data systems that employ cryptographically secure data storage layers, such as blockchain. In many practical applications, blockchain facilitates information sharing. Improved efficiencies in locating and retrieving data across a wide range of data storage tiers will help drive further adoption of such secure data systems.

Description of the drawings

Figure 1 illustrates an example secure decentralized data processing and storage system.

Figure 2 shows an example of the secure data processing and storage node of Figure 1.

Figure 3 illustrates the blockchain stored in the node of Figure 1.

Figure 4 illustrates a secondary indexing system for data elements in a blockchain.

Figure 5 shows the logic flow for creating and updating secondary indexes.

Figure 6 shows another logical flow for creating and updating a secondary index.

Figure 7 shows a timeline for creating and updating secondary indexes.

Figure 8 illustrates the logic flow for querying data elements in the secondary index and blockchain.

Figure 9 shows another logical flow for querying data elements in the secondary index and blockchain.

Detailed ways

Traditionally, data elements describing relationships between entities, such as data related to the exchange of money or services, were maintained and processed by third parties such as banks. Therefore, the integrity of data elements relies on the trustworthiness of third parties and how these data elements are protected from tampering, for example, in servers controlled by third parties. Various data storage layers (such as blockchain systems) have recently been developed to decentralize the processing and storage of these data elements among a large group of untrusted nodes. The following description uses the blockchain reference as an example implementation context, but technical solutions apply to other data storage layers.

Blockchain systems eliminate the need for a trusted third party. In a blockchain system, data elements are stored as publicly accessible data units in the form of a chain of data blocks (i.e., a blockchain) that have multiple copies distributed to various blockchain nodes . Data integrity in blockchain systems is achieved using digital signatures, consensus mechanisms, and other anti-data tampering algorithms. However, the blockchain storing these data elements can be organized using different principles than, for example, traditional relational or non-relational databases. Unlike these traditional databases, which are designed for fast information retrieval, querying the blockchain directly for information can be an inefficient and time-consuming process that involves traversing the entire chain of data blocks. In the present disclosure, auxiliary indexing integrated with blockchain systems provides a technical solution for fast information search and retrieval from data storage layers including blockchain.

Figure 1 illustrates an example system 100 for secure and decentralized data processing and storage, such as a blockchain platform. System 100 includes distributed nodes 102, 104, 106, 108, and 110 that communicate with each other via communication network 120. Communications network 120 may be based on, for example, Internet Protocol (IP), and may include a combination of wired or wireless access networks, local area networks, wide area networks, and other computer networks.

Each node of system 100 may be a combination of software and hardware used, for example, to store, maintain, update, process, and query protected data (hereinafter referred to as a blockchain). Each node can be based on a single computer, a set of centralized or distributed computers, or a single or set of virtual machines hosted by a cloud computing service provider.

As an example, in FIG. 2 , nodes of secure data processing and storage system 100 , such as node 102 , are shown as including a set of computers 201 , such as computers 203 , 205 and 207 . Computer 201 may include a communications interface 202, system circuitry 204, an input/output (I/O) interface 206, a storage device 209, and display circuitry 208 that may be used locally or for remote display (e.g., on a local or remote machine). (in a web browser running on the computer) to generate the machine interface 210. Machine interface 210 and I/O interface 206 may include GUIs, touch-sensitive displays, voice or facial recognition input, buttons, switches, speakers, and other user interface elements. Additional examples of I/O interfaces 206 include microphones, video and still image cameras, headphone and microphone input/output jacks, universal serial bus (USB) connectors, memory card slots, and other types of inputs. I/O interfaces 206 may further include magnetic or optical media interfaces (eg, CDROM or DVD drives), serial and parallel bus interfaces, and keyboard and mouse interfaces.

Communication interface 202 may include a wireless transmitter and receiver ("transceiver") 212 and any antenna 214 used by the transmit and receive circuitry of transceiver 212 . Transceiver 212 and antenna 214 may support Wi-Fi network communications, for example, under any version of IEEE 802.11 (eg, 802.11n or 802.11ac). Communication interface 202 may also include a wired transceiver 216. The wired transceiver 216 may provide a physical layer interface for any of a wide range of communication protocols, such as any type of Ethernet, Data over Cable Services Interface Specification (DOCSIS), Digital Subscriber Line (DSL), Synchronous Optical network (SONET) or other protocols. The computer 201 of the node 102 communicates with other nodes of the blockchain system 100 via the communication interface 202 and the communication network 120.

Memory 209 may be used to locally store a copy of node 102's blockchain. The storage device 209 may further be used to store an auxiliary index of the blockchain. Alternatively, computer 201 may communicate via communication interface 202 and communication network 120 with network storage 230 for storing a copy of the blockchain. Network storage 230 may be centralized or distributed. For example, network storage device 230 may be hosted remotely by a cloud computing service provider.

System circuitry 204 may include hardware, software, firmware, or other circuitry in any combination. System circuitry 204 may be implemented, for example, with one or more systems on a chip (SoC), application specific integrated circuits (ASICs), microprocessors, discrete analog and digital circuits, and other circuitry. System circuitry 204 may include injectable circuitry that interacts with the blockchain and other portions of the system 100 for implementing any desires related to maintenance, storage, processing, verification, indexing, and other aspects of the blockchain. Feature. By way of example only, system circuitry 204 may include one or more instruction processors 218 and memory 220 . Memory 220 stores, for example, control instructions 224 and operating system 222. In one implementation, processor 218 executes control instructions 224 and operating system 222 to implement any desired functionality related to the blockchain. The blockchain's control instructions 224 may be implemented as a software stack with multiple layers.

Returning to FIG. 1 , entities 142 , 144 , 146 , 148 , and 150 may participate in blockchain system 100 via the blockchain nodes of FIG. 1 . In one implementation, each node of blockchain system 100 may support one participating entity. In another implementation, each node may support multiple participating entities or multiple users of participating entities. In order to become a node of system 100, a blockchain software stack can be installed on a computer at that node. At the top of the software stack, the application layer can provide various blockchain functionality supported by lower layers of the software stack. These functions may include, for example, encrypting data elements, submitting data elements for insertion into the blockchain, validating data elements for submission, creating new data blocks in the blockchain via consensus mechanisms, storing local copies of the blockchain and other features.

As shown in Figure 1, the configuration and functionality of blockchain system 100 may be governed by blockchain protocol 130. The blockchain protocol 130 may define how participating entities encrypt data elements. The blockchain protocol 130 may further specify the format of the encrypted data elements so that the encrypted data elements can be decrypted and understood by participating entities and nodes in the blockchain system. The blockchain protocol 130 may additionally specify the types of functions described above and how they should be implemented. The blockchain protocol 130 may additionally specify possible roles of participating entities and nodes. For example, a blockchain node can participate as a fully functional node capable of performing all blockchain functions. Alternatively, a blockchain node may participate in performing only one of a subset of blockchain functions. Each subset of block functions can contain multiple blockchain functions from available blockchain functions. A possible subset of blockchain functionality may be specified in the blockchain protocol 130 . A participating entity may select one of the subsets and install a corresponding software stack for performing functions included in the selected subset. In another implementation, the blockchain protocol 130 may allow participating entities to select any combination of functionality and configure their nodes accordingly, rather than following one of the prescribed subsets of the blockchain protocol 130 .

FIG. 3 shows an example copy of the blockchain 300 stored in a node of the blockchain system 100 that has the function of storing a local copy of the blockchain. As shown in Figure 3, a copy of the blockchain may include a series of linked data blocks 302, 304, 306, 308, and 310, each data block uniquely identified by a block ID (B0, B1, B2, and Bn) . Each data block may include multiple data elements, such as T10, T11, and T12 of block 304, and Tn0, Tn1, and Tn2 of block 310. In addition to data, each data element may also include a data ID. Block 302 may be a special data block or origin block that serves as the head of the blockchain and may not contain actual data elements. Data elements in a data block can include any type of data that participating entities wish to store in the blockchain. For example, a data element may be of a type that describes a relationship between entities, such as a transaction of money or services between two parties. As another example, data elements may include computer instructions for automatically executing the provisions of an agreement or contract embedded in the computer instructions and between the entities. This type of data elements stored in the blockchain may be called executable data elements or smart contracts. As described in greater detail below, execution of one or more segments of computer instructions in an executable data element may be invoked by and from other data elements in the blockchain.

Various cryptographic techniques can be used to achieve the authenticity of the data elements in each data block. For example, digital signatures based on public and private key cryptography can be used to ensure that data elements to be inserted into the blockchain actually come from the submitting entity for which they are announced. In particular, each entity that participates in a blockchain system and wishes to store data elements in the blockchain can possess a private key, which is always kept secret. The public key can be derived from the private key and can be made publicly available. When an entity wishes to store a data element in the blockchain, the entity can first encrypt the data element using a private key before submitting the data for insertion in the blockchain. Anyone with access to the entity's public key can decrypt encrypted data elements. When using public key decryption, any tampering with the encrypted data will render the data unreadable. As such, encryption using a private key represents an entity's digital signature of a data element, and any tampering with the encrypted data is easily detectable.

The data blocks 302, 304, 306, 308, and 310 of the blockchain 300 are created sequentially and linked into a chain. In one example implementation and as shown in Figure 3, the link between a data block and its immediately preceding data block may be a hash value rather than a traditional pointer in the data structure. In particular, a data block may be linked to its immediately preceding data block by including a hash of the data element in the immediately preceding block (referred to herein as a link hash). For example, in Figure 3, hash value Hash 1 of the data element in block 304 may be included in data block 306 immediately following block 304 as a linked hash value. In this way, data block 306 and data block 304 are linked. The algorithm used to calculate the hash value of the data elements contained in the block may be based on, for example, but not limited to, SHA256 hashing.

As shown in Figure 3, the signature code can be further used to sign each data block of the blockchain system containing data elements. The signature code may alternatively be called a one-time nonce. The signature code of the data block is used to help detect tampering of the data block according to the blockchain protocol 130 . For example, the data block signature code may be considered valid according to the blockchain protocol 130 when the hash value of the combination of the signature code and the data elements in the data block contains a hash portion with a predefined hash pattern. A predefined hash pattern may be specified by the blockchain protocol (eg, a predefined number of leading zeros at the beginning of the hash value). Therefore, the signature code for a block can be calculated by solving difficult cryptographic problems. In the above example implementation, the signature code may be calculated such that the calculated hash value of the combination of the signature code and the data element conforms to the signature protocol. For example, signature code 1 of block 304 is calculated such that the SHA256 hash of the combination of signature code 1 of data block 304, data elements T10, T11, and T12 is compatible with the block signing protocol, e.g., by a predefined number of zeros leading . Any tampering with the data in a signed block will cause the hash of that block to be incompatible with the signing protocol.

Blockchain 300 can be created by appending data blocks one at a time. Specifically, encrypted (or digitally signed) data elements from various nodes of the blockchain system may be broadcast to the blockchain system. These encrypted data elements can then be collected into data blocks for storage into the blockchain. The blockchain protocol 130 may specify a consensus algorithm or mechanism. The consensus algorithm controls how new data elements are verified, how new data blocks are assembled from verified new data elements, and how the new block is broadcast to the blockchain nodes, checks it, and accepts it into the blockchain.

In an example consensus algorithm, data elements submitted from nodes may be collected periodically (e.g., every 10 minutes or approximately the period of time required by a blockchain node to solve a block signing code). Further, various nodes may perform verification according to the data element verification rules specified by the blockchain protocol 130 (e.g., the data element is digitally signed, the financial transaction contained in the data element is valid because the payer has a sufficient balance) Participates in validating these data elements to assemble the data elements into a block, calculates the signature code for the block, and broadcasts the new block to the blockchain system for acceptance. In one implementation, the first node to broadcast an acceptable block would be responsible for inserting its block into the blockchain. Update copies of the blockchain in various nodes with new blocks. This specific algorithm that must solve the signed code (sometimes called "proof of work") is just one example of a possible consensus algorithm. Blockchain annotations can use other consensus algorithms (such as "proof of stake") to verify and create new data blocks. Alternatively, inserting a data element embedded in its data block into the blockchain is called linking the data element with the blockchain.

Functions of a blockchain system, such as encrypting and submitting data elements, consensus functions including verification of data elements, computation of signature codes, and assembly of new data blocks, and storing local copies of the blockchain, can be performed by various nodes. implement. Nodes participating in the consensus algorithm may be called miners, for example. As mentioned before, nodes can decide on a subset of functions when participating in a blockchain system, and nodes can execute that subset of functions by installing the corresponding software stack. For example, a fully functional node can perform all the functions discussed above. However, function-restricted nodes can only perform a selected set of functions. For example, some nodes can only participate in encrypting data elements and submitting them to the blockchain.

As mentioned above, data elements in a blockchain's data blocks can be any type of data that participating entities wish to store in the blockchain, including special types of executable data elements (or smart contracts). Blockchain protocol 130 (130 of Figure 1) may provide for mechanisms and interfaces for calling all or part of executable computer instructions in executable data elements from other data elements. For example, executable data elements may include portions of instructions that may be independently invoked. Each of these executable data elements may be identified by an ID, and each portion of the instructions within the executable data element may further be identified by a portion ID.

Execution of all or part of an executable data element may be invoked in various ways and at various timings. In one example implementation, the blockchain protocol 130 may provide a mechanism for when another data element is inserted into a new data block and the new data block is verified and appended by one of the nodes. When to the blockchain, execution of the portion of the instructions in the executable data element is invoked from that other data element. A call interface may be provided in a call data element in accordance with the blockchain protocol 130 for specifying, for example, an ID of an executable data element, an ID of a particular portion of an instruction within the executable data element to be executed, and parameters passed to the executable instruction. .

The blockchain system described above in Figures 1 to 3 can be used in a variety of applications. For example, multiple participating entities can use a blockchain system to share proprietary information stored in the blockchain. For example, collaborating semiconductor circuit manufacturers could use a blockchain system to share designs and layouts. As another example, two partner hospitals could use a blockchain system to share patient activity and other information. The hospital can classify the information into, for example, different types of patient activity, such as patient activity and interactions with different departments (internal medicine, cardiology, payment department, etc.). Each individual campaign data can be proprietary and encoded by the hospital according to a format and encryption formula agreed upon between the collaborating hospitals and used as the data payload in the data element. This data element can then be further encrypted using the appropriate public key and stored in the blockchain. As such, proprietary information in this data element can only be shared among collaborating hospitals and not with other general participating entities in the blockchain. Specifically, while other participating entities can use the public key to decrypt data elements in the blockchain to obtain an encapsulated data payload, this data payload is proprietary and can only be further decrypted by the collaborating hospital.

In the above context of information sharing in blockchain and many other blockchain applications, participating entities may desire to query the blockchain for information. For example, one of the partner hospitals may need to search for patient activity in a specific department of another hospital. For a typical blockchain system, specific data elements are usually obtained directly from the blockchain if the data ID and/or block ID of the data block containing the specific data element is known and used as a query key. The returned data elements can then be decrypted using the appropriate public key, and if the data payload is proprietary, it can be further decrypted by the querying entity.

However, in some applications, the data ID and/or block ID of the data element to be queried may not be known. For example, in the information sharing application between two collaborating hospitals discussed above, the first hospital may wish to find out where a specific patient visited a specific department of the second hospital. The keys that First Hospital can use to query the blockchain could be the identity of a specific department of the hospital and the name or public key of a specific patient. Because the block ID and data ID of the data elements associated with these patient visits are unknown, it may be necessary to traverse the blockchain from beginning to end in order to decrypt and identify these expected data elements stored in the blockchain. This query process in the blockchain can be time-consuming and inefficient because the blockchain can become too large.

In some other implementations, information about the entire collection of data elements in the blockchain can be extracted and tracked as blocks of data are appended to the blockchain in the form of a separate database, such as a relational database, which can be used Traditional database query process to query more efficiently. However, this approach requires additional storage space in the blockchain node for maintaining and synchronizing separate databases. Such databases duplicate information already contained in the blockchain and typically consume large storage space.

In one example implementation in accordance with the present disclosure, to efficiently retrieve information from a blockchain, one entity or collaborating entities may create and update a secondary index for a subset of data elements in the blockchain. The subset of data elements may include data elements of interest to the entity or collaborating entity. For example, a secondary index may be established by two collaborating hospitals that wish to share information about data elements related to patient activity in these collaborating hospitals. Each auxiliary index can correspond to a specific type of patient activity. Auxiliary indexes may be created and updated via execution of instructions in executable data elements submitted to the blockchain by partner hospitals. When each subset of data elements of a specific type is inserted into the blockchain, execution of the executable data elements that creates and updates the secondary index can be invoked. Further, the secondary index can be maintained in a database separate from the blockchain. Both hospitals have access to secondary indexes to facilitate retrieving information from the blockchain regarding subsets of data elements without having to traverse the entire blockchain.

Figure 4 illustrates an example implementation 400 of such an auxiliary index for information lookup in a blockchain in greater detail. Specifically, executable data element 402 may be inserted into the blockchain following the normal process of submitting data elements according to blockchain protocol 130 . For example, these executable data elements may be agreed to by entities wishing to share information. These executable data elements may be executed to create, maintain, update, and query the auxiliary index 422 for facilitating the entity's retrieval of information. Each executable data element may correspond to a secondary index (eg 404→424, 406→426, 408→428, 410→430). Alternatively, the executable data element may contain multiple portions of the instruction, and each portion of the instruction may correspond to one of the auxiliary indices 422 .

Each executable data element and corresponding secondary index can be related to a specific type of data element in the blockchain. Accordingly, auxiliary indexes 422 may each be identified as being associated with a type of data element. For example, in the context of an information sharing application between two collaborating hospitals, auxiliary index 424 may correspond to all patient activities of the hospital's Department of Cardiology, while index 426 may relate to all patient activities of the hospital's Department of Surgery, and index 428 may Corresponds to payments made by patients to the hospital.

In one implementation, each auxiliary index, such as auxiliary index 426, may include a public key for the entity involved in the data element of the corresponding type as shown at 440. Further, each index may include a block ID and/or a data ID of a corresponding type of data element. In the context of an information sharing application between two collaborating hospitals, the auxiliary index 426 may include multiple entries, for example. Each entry may correspond to a data element related to patient activity in, for example, surgery in a hospital. Each entry may include the patient's public key or other identifier as well as the block ID (eg, b11-b1N) and data ID (eg, t11-t1N) of the data element, as shown at 440, 450, and 460 of Figure 5.

Figure 5 illustrates the logic flow for creating secondary indexes for specific types of data elements. An entity may first submit an executable data element to the blockchain that contains instructions for building a secondary index for a particular type of data element (502). The executable data elements may then be included into the blockchain by the miner nodes via a consensus process (504). Once executable data elements are in place in the blockchain, secondary indexes can be built for specific types of data elements. Specifically, data elements of specific types may be created (506). Specific types of data elements may then be submitted to and included in the blockchain by the miner nodes via a consensus process (508). When a particular type of data element is included in the blockchain, instructions in the executable data element for establishing a secondary index for the particular type of data element may be invoked and automatically executed (510). This call may be performed by including a code interface in the specific type of data element for automatically calling instructions in the executable data element when the specific type of data element is submitted to the blockchain. The code interface may be specified by the blockchain protocol 130. Alternatively, the invocation of the instructions in the executable data element for building the secondary index may be made in the application layer of the node. By executing the auxiliary index building instructions, the data ID, block ID, public key or other entity identifier, etc. may be extracted from the specific type of data element and included in the auxiliary index of the specific type of data element (512).

Figure 6 illustrates another alternative logic flow for creating an auxiliary index for a specific type of data element. Again, the entity may first submit an executable data element to the blockchain that contains instructions for building a secondary index for the particular type of data element (602). The executable data elements may then be included into the blockchain by the miner nodes via a consensus process (604). Once the executable data elements are in place, secondary indexes can be built for specific types of data elements. Specifically, specific types of data elements may be created (606) and included into the blockchain by miner nodes via a consensus process (608). When including data elements into the blockchain, a corresponding index data element (alternatively referred to as an index control data element) may be created with an interface for calling indexing instructions within executable data times (610). The creation of index data elements can be initiated automatically by a code interface embedded in the data element. Alternatively, the application layer in the blockchain node can create the index data element after detecting that a corresponding data element of a specific type has been submitted to the blockchain. The index data element is just like a regular data element, but contains an interface for calling instructions in the executable data element that are used to build secondary indexes for specific types of data elements. The index data elements are then included into the blockchain by the miner nodes via a consensus process (612). When an index data element is included in the blockchain, instructions in the executable data element for building a secondary index for a particular type of data element may be invoked and automatically executed (614). By executing the auxiliary index building instructions, the data ID, block ID, public key or other entity identifier, etc. may be extracted from the specific type of data element and included in the auxiliary index of the specific type of data element (616).

Figure 7 shows an example timeline for creating secondary indexes for specific types of data elements. Specifically, FIG. 7 shows a timeline 701 with three time markers t1, t2, and t3, where t1 < t2 < t3. Before time t1, entities such as two collaborating hospitals H1 and H2 have created various executable data elements for building secondary indexes for specific types of data elements. These executable data elements may be included in block 702, for example. In Figure 7, two separate executable data elements 710 and 711 are included in block 702 for establishing auxiliary indexes for patient visits to the cardiology and surgery departments of the hospital, respectively. Alternatively, the two executable data elements can reside in different data blocks. Alternatively, the two separate executable data elements may be implemented as a single executable data element that contains two identifiable portions of instructions, one to build a secondary index for patient visits to the cardiology department of the hospital, and Another part is used to build an auxiliary index for patient visits to the hospital's surgical department.

At time t1, the data block 704 is verified by the miner node through the consensus process and accepted into the blockchain. Among various data elements, data block 704 contains data element 712 describing first patient P1's visit to H1's cardiology department. Data element 712 may directly invoke instructions in corresponding executable data element 710 in block 702 to create entry 722 in cardiology assist index 720 . Alternatively, data element 712 may create index data element 713 that invokes instructions in executable data element 710 to create entry 722 in cardiology assist index 720 . Therefore, after time t1, the cardiology ancillary index 720 contains one entry, and the surgery index 730 is empty. Data element 712 and index data element 713 need not be in the same data block. For example, index data element 713 may be in a later data block, such as the data block immediately following the data block holding data element 712.

At time t2, the data block 706 is verified by the miner node through the consensus process and accepted into the blockchain. Among various data elements, data block 706 contains data element 714 describing second patient P2's visit to H2's surgery. Again, data element 714 may directly invoke instructions in corresponding executable data element 711 in block 702 to create entry 724 in surgical assistance index 730 . Alternatively, data element 714 may create index data element 715 that invokes instructions in executable data element 711 to create entry 724 in surgical assistance index 730 . Therefore, after time t2, the number of entries in the cardiology ancillary index 720 remains one, and the surgical ancillary index 730 contains one entry.

At time t3, the data block 708 is verified by the miner node through the consensus process and accepted into the blockchain. Among the various data elements, data block 708 contains data element 716 describing third patient P3's cardiology and surgery visit to H2. Data element 716 may directly call instructions in two executable data elements 710 and 711 in block 702 to create entry 726 in cardiology assist index 720 and entry 728 in surgical assist index 730 . Alternatively, data element 716 may create index data element 717 that invokes instructions in executable data elements 710 and 711 to create entry 726 in cardiology assist index 720 and entry 728 in surgical assist index 730 . Therefore, after time t3, the cardiology ancillary index 720 contains two entries, and the surgical ancillary index 730 contains two entries.

In another example context of information sharing between two collaborating semiconductor manufacturers, the semiconductor manufacturer may store circuit layouts designed for various equipment manufacturers (customers of the semiconductor manufacturer) in the blockchain. To name a few examples, circuits can be classified into different types such as analog power circuits, digital microcontroller circuits, digital signal processor circuits, encryption/decryption circuits, video encoding circuits. According to the above principles, semiconductor manufacturers can create various executable data elements (one executable data element per circuit) for establishing auxiliary indexes for data elements containing these types of circuits. Thus, for example, auxiliary index 720 of FIG. 7 may be established for data elements containing analog power circuits, while index 730 may be established for data elements containing digital signal processor circuits. Correspondingly, executable data elements 710 and 711 may contain instructions for establishing auxiliary indexes for analog power circuits and digital signal processor circuits. Additionally, data element 712 may contain an analog power circuit submitted by the first semiconductor manufacturer to the first device manufacturer, and corresponding index data element 713 may be used to invoke instructions in executable data element 710 to build auxiliary index 720 . Data element 714 may contain digital signal processor circuitry submitted by the second semiconductor manufacturer to the second device manufacturer, and corresponding index data element 715 may be used to invoke instructions in executable data element 711 for establishing the Auxiliary index 730 for digital signal processor circuits. Similarly, data element 716 may contain both analog power circuitry and digital signal processor circuitry submitted by the second semiconductor manufacturer to the third device manufacturer, and corresponding index data element 717 may be used to invoke the two executables. Instructions in data elements 710 and 711 for establishing auxiliary indexes 720 and 730 for analog power circuits and digital signal processor circuits. As such, the diagram in Figure 7 is suitable for use in the context of information sharing between collaborating semiconductor manufacturers and in many other application contexts.

Auxiliary indexes 720 and 730 may be stored separately from the blockchain. They can be stored locally at nodes belonging to the hospital. Alternatively, they can be stored in cloud storage space. Access to the secondary index may be provided to the hospital and entities to which the hospital agrees to grant access. Secondary indexes can be stored in the form of a relational database or any other form. Alternatively, the secondary index can be maintained in any other form and can be optimized for queries. For example, the secondary index can be stored as a data structure such as a binary sorted tree, and can be optimized using optimization algorithms involving hashing. Because there is no need to write the auxiliary index back to the blockchain, it does not affect the verification and consensus process of the main blockchain. In this way, all consensus mechanisms used in traditional blockchains remain intact, and these consensus mechanisms can be further developed independently of the development of mechanisms used to build secondary indexes.

Further, individuals or collaborating entities in the application layer in the blockchain node can develop instructions for establishing auxiliary indexes. The lower layers of the blockchain software stack only need to provide mechanisms or interfaces for function calls to executable data elements (smart contracts) in the blockchain from another data element in the blockchain. As such, the process described above for building and updating the secondary index does not require any modifications to the underlying software stack at the blockchain node.

Because the secondary index is created and updated according to Figures 5 and 6 by encapsulating the instructions used to build the secondary index in an executable data element in the main blockchain, the secondary index is ensured by the consensus mechanism for the main blockchain Updated for completeness. Further, the auxiliary index only contains identifier information for entities, data elements, and blocks. The actual data only resides in the main blockchain and is protected from tampering. As such, any tampering with the secondary index (which may reside outside the main blockchain and thus not protected by the main blockchain's tamper detection mechanisms) will at most result in misidentification when querying the secondary index for data IDs and block IDs. Data elements or missing data elements (see below for a detailed description of the query process). The actual data elements in the main blockchain will not be compromised.

There may be a time delay between submitting a specific type of data element to the blockchain and updating the secondary index for the specific type of data element. Because of this delay, the data elements and corresponding index data elements of Figures 6 and 7 (such as data element 712 and corresponding index data element 713) may reside in different blocks. However, because the instructions used to build the secondary index are executed after the data elements have been committed to the main blockchain, any entries in the secondary index will correspond to data elements that are already included in the main blockchain. The instructions in the executable data element for building the secondary index may be further structured to ensure that the secondary index is not updated until the corresponding data element is committed to the blockchain across blockchain nodes. Alternatively, the application layer in the blockchain node may ensure that the index data element is submitted to the blockchain after the corresponding data element is submitted to the blockchain (see 610 of Figure 6).

Figure 8 illustrates the logic flow for querying information in the blockchain via secondary indexes established for specific types of data elements. An entity, such as one of the collaborating hospitals, may submit an executable data element containing instructions for querying a secondary index for a desired type of data element (802). This executable data element may be integrated with executable data element 502 of FIG. 5 or 602 of FIG. 6 . Specifically, an executable data element may contain portions of instructions for creating an auxiliary index for a particular type of data element and portions of instructions for querying the auxiliary index. Alternatively, executable data element 802 of FIG. 8 may be separate from executable data element 502 of FIG. 5 or 602 of FIG. 6 .

Continuing with Figure 8, executable data elements for querying information about specific types of data elements may then be included in the blockchain by the miner nodes via a consensus process (804). Once the executable data element is in place, the entity may query the auxiliary index by directly invoking instructions for querying the auxiliary index for a particular type of data element (806). For example, query instructions can be executed to query a secondary index for a specific type of data element using a patient's name or public key. This direct query can be independent of the blockchain. For example, the secondary index can be stored in a database outside the blockchain, and queries can be made directly by the entity from its blockchain node or from anywhere outside the blockchain node. A query on the secondary index may return output containing block IDs and/or data element IDs that match the query criteria (808). The entity may then use the returned block ID and/or data ID to obtain corresponding data elements from the blockchain, decrypt the data elements using the public key, and further decode the data payload to obtain the desired information (810).

Figure 9 illustrates another alternative logic flow for querying information in the blockchain via secondary indexes established for specific types of data elements. An entity, such as one of the collaborating hospitals, may submit an executable data element containing instructions for querying a secondary index for a desired type of data element (902). Again, this executable data element may be integrated with executable data element 502 of FIG. 5 or 602 of FIG. 6 . Executable data elements for querying information about specific types of data elements may then be included in the blockchain by the miner nodes via a consensus process (904). Once the executable data element is in place, the entity can query the secondary index by using a query data element that contains an interface for invoking query instructions for the executable data element (906). The query data elements may first be included in the blockchain by miner nodes using a consensus process (908). Instructions for querying a secondary index for a particular type of data element in the executable data element may be automatically invoked upon inclusion of the query data element into the blockchain (910). For example, query instructions can be executed to query a secondary index for a specific type of data element using a patient's name or public key. A query on the secondary index may return output containing block IDs and/or data element IDs that match the query criteria (912). The entity may then use the block ID and/or data ID to obtain the corresponding data elements from the blockchain, decrypt the data elements using the public key, and further decode the data payload to obtain the desired information (914).

Accordingly, the present disclosure provides a mechanism for establishing secondary indexing of subsets of data elements in a blockchain. Auxiliary indexes can be built and updated at the application layer without affecting the underlying software stack of the blockchain node. Auxiliary indexes can be maintained outside the blockchain. Auxiliary indexes facilitate efficient querying of data elements in blockchains, especially in the context of information sharing in blockchain systems. While the above disclosure is primarily made in the context of blockchain, the underlying principles apply to any decentralized secure data processing and storage system.

The methods, devices, processes and logic described above may be implemented in many different ways and in many different combinations of hardware and software. For example, all or part of the implementation may be circuitry including: an instruction processor, such as a central processing unit (CPU), a microcontroller, or a microprocessor; an application specific integrated circuit (ASIC), a programmable logic device (PLD) ) or a field programmable gate array (FPGA); or a circuit device that includes discrete logic or other circuit components, including analog circuit components, digital circuit components, or both; or any combination thereof. As examples, the circuitry may include discrete interconnect hardware components, and/or may be combined on a single integrated circuit chip, distributed among multiple integrated circuit dies, or multiple integrated circuit dies in a common package is implemented in a multi-chip module (MCM).

The circuitry may further include or access instructions for execution by the circuitry. Instructions may be stored in a tangible storage medium other than a transient signal, such as flash memory, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM); or stored in On a magnetic or optical disk, such as a compact disk read-only memory (CDROM), hard disk drive (HDD), or other magnetic or optical disk; or stored in or on another machine-readable medium. A product, such as a computer program product, may include a storage medium and instructions stored in or on the medium, and the instructions, when executed by circuitry in the device, may cause the device to perform any of the processes described above or illustrated in the figures.

These implementations may be distributed as circuitry among multiple system components, such as among multiple processors and memories, optionally including multiple distributed processing systems. Parameters, databases, and other data structures can be stored and managed separately, can be combined into a single memory or database, can be logically and physically organized in many different ways, and can be implemented in many different ways, including As a data structure such as a linked list, hash table, array, record, object or implicit storage mechanism. A program may be part of a single program (e.g., a subroutine), a separate program, distributed across multiple memories and processors, or implemented in many different ways, such as in a shared library such as a dynamic link library (DLL). in the library. For example, a DLL may store instructions that, when executed by circuitry, perform any of the processes described above or illustrated in the figures.

Various implementations have been described in detail. However, many other implementations are possible.

Claims (20)

1. An electronic system, comprising:

a memory configured to store:

a data storage layer comprising linked data blocks comprising data elements; and

an index structure comprising index entries for data elements of a predefined type in the linked data block; and

injection circuitry in communication with the memory, the injection circuitry configured to respond to a link of a new data block comprising a new data element to the linked data block in the data storage layer by:

determining that the new data element is of the predefined type;

generating an index entry for the new data element, the index entry including a location reference within the data storage layer for the new data element; and

the index entry is inserted into the index structure.

2. The system of claim 1, wherein the data storage layer resides within a blockchain system node and the linked data blocks constitute copies of blockchains.

3. The system of claim 1, wherein the data elements comprise executable data elements comprising instructions to generate the index entry and insert the index entry into the index structure.

4. A system according to claim 3, wherein the injection circuitry is configured to invoke the instructions in the executable data element from the new data element to generate the index entry and insert the index entry into the index structure after the new data element is linked with the linked data block in the data storage layer.

5. The system of claim 3, wherein,

the injection circuitry is further configured to generate an index control data element for the new data element and link the index control data element to the linked data block in the data storage layer; and is also provided with

The injection circuitry is configured to invoke the instruction in the executable data element from the index control data element after the new data element and the index control data element are linked with the data block in the data storage layer.

6. The system of claim 1, wherein the index entry further comprises a data characteristic describing the new data element of the predefined type.

7. The system of claim 6, wherein the location reference comprises a data chunk identifier, a data element identifier, or both.

8. The system of claim 1, wherein the index structure is stored outside of the linked data blocks in the data storage layer.

9. The system of claim 1, further comprising an interrogation circuit configured to:

receiving a query for the new data element of the predefined type of data elements;

identifying the index structure associated with the data element of the predefined type;

retrieving the index entry from the index structure; and

in response to the query, the location reference obtained from the index entry is returned.

10. The system of claim 9, wherein the location reference comprises the location reference within the data storage layer for the data element of the predefined type.

11. The system of claim 9, wherein the data elements comprise executable data elements comprising instructions to identify the index structure and retrieve the index entry.

12. The system of claim 11, wherein the query is embedded in a query data element, and the query circuitry is further configured to invoke the instruction in the executable data element from the query data element after the query data element is linked with the linked data block.

13. A computer-implemented method, comprising:

receiving an executable data element comprising instructions for generating an index entry for a data element of a predefined type and inserting the generated index entry into an index structure for the predefined type;

linking the executable data elements with linked data blocks in a data storage layer;

receiving new data elements of the predefined type;

linking the new data element with the linked data block in the storage layer; and

invoking the instructions in the executable data elements linked with the linked data block to generate an index entry for the new data element and insert the index entry into the index structure after the linking of the new data element with the linked data block,

Wherein the index entry includes a location reference within the data storage layer for the new data element.

14. The method of claim 13, wherein the instruction in the executable data element is invoked from the new data element.

15. The method of claim 13, further comprising:

generating an index control data element for the new data element; and

linking the index control data element with the linked data block in the storage layer, and wherein the instructions in the executable data element are invoked from the index control data element.

16. The method of claim 13, wherein the index structure is stored outside of the linked data blocks in the data storage layer.

17. The method of claim 13, further comprising:

receiving a query for the new data element of the predefined type of data elements;

identifying the index structure associated with the predefined type;

retrieving the index entry from the index structure; and

in response to the query, the location reference obtained from the index entry is returned.

18. The method of claim 17, wherein the instructions in the executable data element further comprise instructions for the identification of the index structure and the retrieval of the index entry.

19. The method of claim 18, wherein the query is embedded in a query data element, the method further comprising:

linking the query data element with the linked data block in the storage layer; and

invoking the instructions in the executable data element for identifying the index structure and retrieving the index entry from the query data element after the query data element is linked with the linked data block in the storage layer.

20. An electronic system, comprising:

a memory configured to store:

a data storage layer comprising linked data blocks comprising data elements; and

an index structure comprising index entries for data elements of a predefined type in the linked data block; and

injection circuitry in communication with the memory, the injection circuitry configured to:

Linking executable data elements containing instructions for generating index entries and inserting the index entries for the predefined type of data elements into the linked data blocks in the data storage layer;

receiving a new data element;

determining that the new data element is of the predefined type;

generating an index control data element for the new data element of the predefined type;

linking the new data element with the linked data block in the data storage layer;

linking the index control data element with the linked data block in the data storage layer; and

generating an index entry for the new data element and inserting the index entry into the index structure by invoking the instruction in the executable data element from the index control data element after the new data element and the index control data element are linked with the linked data block in the data storage layer,

wherein the index entry includes a location reference within the data storage layer for the new data element.