CN110268476A - Systems and methods for managing large medical image data - Google Patents

Abstract

A system and method for managing medical image data items. The large image manager is used to store medical image data items external to the core PACS. Representative objects are generated and provided to PACS to be incorporated into the core PACS workflow. Representative objects can be generated from pixel data and metadata extracted from medical image data items. Representative objects are generated in a format compatible with protocols used by PACS workflows. Requests for medical images sent to the PACS can be rerouted to the large image manager to provide the requested images.

Description

Systems and methods for managing large medical image data

technical field

The described embodiments relate generally to managing medical data, and in particular to systems and methods for managing large medical image data items, systems and methods for collecting medical image data items, and methods for processing medical image data items systems and methods.

Background technique

The following is not an admission that anything discussed below is part of the prior art or part of the common general knowledge of those skilled in the art.

Electronic document management systems are increasingly being used to manage medical records. These systems are used to facilitate and simplify processes such as generating, storing, archiving, searching, reviewing and reporting electronic medical records. As the size of electronic medical records increases, efficient systems for managing and organizing electronic medical records are needed to avoid inefficiencies and backlogs in data transmission and communication. An inefficient or slow document management system can have a direct impact on clinical efficiency, for example by increasing the time it takes clinicians to retrieve and review each patient document.

Picture Archiving and Communication System (PACS) provides storage and management of medical image data items. PACS defines a common Digital Imaging and Communications in Medicine ("DICOM") format for communication (eg, storage and transmission) of image files and associated metadata. PACS systems are used in many healthcare disciplines such as, but not limited to, radiology, computed tomography, positron emission tomography, magnetic resonance imaging, sonography, and cardiology. Therefore, along with other communication protocols such as the Health Level Seven International (HL7) standard and the Fast Healthcare Interoperability Resource (FHIR) standard, the DICOM standard and DICOM messaging play a central role in modern digital healthcare. play a key role in.

Medical disciplines that involve the collection and review of medical images have increasingly transitioned to electronic and digital imaging systems. Pathology is an example of such a discipline. Pathology is the study of disease in human tissue, and pathology results are often used as key test results for diagnosis and treatment decisions (for example, HER2 tissue measurements required to allow the prescribing of certain cancer drugs). Pathology can involve microscopic, macroscopic, chemical (molecular biology) and biological analysis of tissue samples.

In response to increasing workloads, volume pathology departments are digitizing tissue sample slides into high-resolution whole slide images (WSIs). These digitized slides are used to reorganize clinical routines and procedures, and can also be used in clinical research. Laboratory medicine, anatomical pathology, and the fusion of digital imaging and biomarker data are dramatically changing the way pathology and laboratory medicine are practiced. These in vivo and in vitro diagnostics often need to be translated into comprehensive reports with data collected from various sources.

In pathology, the transition to the digital world has been enabled by commercially available digitizing imagers for digitizing microscope slides. Each digitizing imager generates a digital representation of the microscopic slide in one or more data files that together form an electronic medical image data item. However, the transition to electronic medical images and medical image data items is not without technical challenges.

Different medical imagers may generate medical image data items in different and sometimes proprietary formats. While some of these imaging formats are based on open source imaging specifications with proprietary modifications (eg, TIFF, JPEG, BMP, etc.), others use entirely proprietary data formats. As a result, different viewing applications may be required to view medical image data items generated by different viewers.

Generating medical image data items in different formats may also complicate or limit the ability to integrate medical image data items from different imagers into a centralized file management system (such as a PACS) for managing medical images. This may result in electronic workflows being restricted to specific departments or laboratories that generate medical image data items in a format defined by a specific document management system. As a result, electronic workflows may only cover a portion of the necessary steps required for a particular imaging task.

For example, in the case of ultrasound-guided biopsy, the tissue acquisition workflow forms part of the PACS system, whereas the sample processing workflow is external to, and thus invisible to, the PACS. The same may be true for CT/MR or endoscopically guided sampling procedures. As a result, a single imaging procedure may require multiple independent electronic workflows (which may not communicate with one another).

When different imagers and imaging workflows are not integrated into a centralized system, it can be difficult for clinicians to obtain a holistic view of related medical image data items available for the same patient or images of the same type. For example, in the case of breast tissue sampling, samples from the same tissue block can be used both to produce microscope slides and to perform x-ray imaging. However, X-ray images may not be available in the pathology image management system, and pathology may not be available in the X-ray image management system.

Clinicians using one image management system may not have direct access to images from other systems, which can introduce delays as they must change programs or equipment to access the other system. In some cases, the clinician may not be aware that other images have been captured or are available on other file systems, which may lead to oversights or errors.

Contents of the invention

This Summary is intended to introduce the reader to the following more detailed description, not to limit or define any claimed or as yet unclaimed invention. One or more inventions may reside in any combination or subcombination of elements or process steps disclosed in any part of this document, including its claims and drawings.

According to one broad aspect, a system for managing medical image data items is provided. The system includes: a picture archiving and communication system (PACS) including an image storage memory configured to store medical image data items, and the PACS is communicatively coupled to at least one observation station; with the PACS and with at least one medical imager A large image manager component in communication, the large image manager component comprising a large image storage memory; wherein the large image manager component is configured to: identify a large medical image data item generated by at least one medical imager; and store data from the large image storing pixel data for each of the large medical image data items in memory; generating at least one representative data object for each of the large medical image data items; and converting at least one representative data object of each of the large medical image data items transmitting the characteristic data object to the PACS; and the PACS is configured to: store at least one representative data object of each of the large medical image data items in the image storage memory.

In some embodiments, the PACS is configured to: receive a request for a particular medical image from one of the observation stations; determine that the particular medical image corresponds to one of the large medical image data items stored in the large image storage memory; and The request for the particular medical image is rerouted to the large image manager component; and the large image manager component is configured to, in response to the rerouted request, transfer the particular pixel data corresponding to the particular medical image from the corresponding large medical image data item The stored pixel data is provided to the viewing station.

In some embodiments, the large image manager component is further configured to: determine that a new large medical image data item has been generated at one of the medical imagers, the new large medical image data item including defining one or more sub-images pixel data and metadata corresponding to one or more sub-images; extract metadata from the new large medical image data item; generate at least one representative data object corresponding to the new large medical image data item to include the extracted metadata at least a portion of the data; and transmitting at least one representative data object to a PACS; and the PACS is configured to store the at least one representative data object in an image storage memory.

In some embodiments, the large image manager component is configured to: determine that a new large medical image data item has been generated at one of the medical imagers, the new large medical image data item including an pixel data and metadata corresponding to the one or more sub-images; extracting the pixel data from the new large medical image data item; and storing the extracted pixel data in a large image storage memory.

In some embodiments, the large image manager component is further configured to: store the new large medical image data item in an image archive in the large image storage memory; and store the extracted pixel data in the large image storage memory for processing in the data cache.

In some embodiments, the large image manager component is configured to generate at least one representative data object to include address metadata identifying a location of pixel data corresponding to the new large medical image data item; and the PACS is configured to : identifying at least one representative data object in response to receiving a request from one of the observation stations for a particular medical image corresponding to a new large medical image data item; identifying the stored the location of the pixel data for the ; and use the identified location to reroute the request for the specific medical image to the large image manager component.

In some embodiments, rerouting the request for the particular medical image data item includes providing a routing response to the viewing station indicating the identified location of stored pixel data corresponding to the particular medical image.

In some embodiments, rerouting the request for the particular medical image data item includes automatically rerouting, by the PACS, the request to the large image manager component in response to receiving the request for pixel data from the viewing station.

In some embodiments, the PACS is configured to: define a plurality of large medical imager types, the plurality of large medical imager types including medical imagers configured to: generate large medical image data items; Determining from the received request that the particular medical image data item was generated by a medical imager corresponding to one of the larger medical imager types; and in response to determining that the particular medical image data item was generated by a medical imager corresponding to one of the larger medical imager types Generated by medical imagers, automatically rerouting requests to the large image manager component.

In some embodiments, the large medical image data item includes a plurality of sub-images of the series of medical images; and the large image manager component is configured to: store image-specific images for each of the sub-images of the plurality of sub-images in the series of medical images; Pixel data, wherein image-specific pixel data for at least some of the sub-images is stored separately from image-specific pixel data for other parts of the sub-images.

In some embodiments, the received request identifies one of the sub-images of the series of medical images corresponding to the particular medical image; and the large image manager component is configured to provide an image of the requested sub-image in response to the rerouted request specific pixel data.

In some embodiments, the large image manager component is configured to provide pixel data corresponding to a particular medical image directly to the viewing station in response to the rerouted request.

In some embodiments, the large image manager component is configured to provide pixel data corresponding to a particular medical image to the viewing station via the PACS in response to the rerouted request.

In some embodiments, the PACS is configured to communicate using a PACS communication protocol; the medical imager is configured to generate large medical image data items in a format different from that defined by the PACS communication protocol; and a large image manager component is configured to generate at least one representative data object in a format defined by the PACS communication protocol.

According to a broad aspect, there is provided a method of managing medical image data items using a picture archiving and communication system (PACS) comprising an image storage memory configured to store medical image data items and associated with a large Large image manager component for PACS communication of image storage memory. The method includes: identifying large medical image data items generated by at least one medical imager; storing pixel data corresponding to each of the large medical image data items in a large image storage memory; Each of the large medical image data items generates at least one representative data object; the large image manager component transmits the at least one representative data object of each of the large medical image data items to the PACS; and the large medical image data At least one representative data object for each of the items is stored in the PACS.

In some embodiments, the method further includes: receiving, by the PACS, a request for a specific medical image from an observation station; determining that the specific medical image corresponds to one of the large medical image data items stored in the large image storage memory; The request for the image is rerouted to the large image manager component; and in response to the rerouted request, the specific pixel data corresponding to the specific medical image is provided by the large image manager component from the pixel data stored for the corresponding large medical image data item to the observation post.

In some embodiments, the method further comprises: determining that a new large medical image data item has been generated at one of the medical imagers, the new large medical image data item comprising pixel data defining one or more sub-images and metadata corresponding to the one or more sub-images; extracting the metadata from the new large medical image data item; generating at least one representative data object corresponding to the new large medical image data item to include at least a portion of the extracted metadata ; and storing at least one representative data object in the PACS.

In some embodiments, the method further comprises: determining that a new large medical image data item has been generated at one of the medical imagers, the new large medical image data item comprising pixel data defining one or more sub-images and corresponding metadata for the one or more sub-images; extracting pixel data from the new large medical image data item; and storing the extracted pixel data in the large image storage memory.

In some embodiments, the method further comprises: storing the new large medical image data item in an image archive in the large image storage memory; and wherein the extracted pixel data is stored in a processed data cache in the large image storage memory . In some of these embodiments, extracted metadata may also be stored in a processing data cache.

In some embodiments, the method further comprises: generating at least one representative data object to include address metadata identifying the location of pixel data corresponding to the new large medical image data item; For a request for a specific medical image corresponding to a new large medical image data item, at least one representative data object is identified by the PACS; the location of the stored pixel data is identified by the PACS from the address metadata in the at least one representative data object; and The identified location is used by the PACS to reroute requests for specific medical images to the large image manager component.

In some embodiments, rerouting the request for the particular medical image data item includes providing a routing response to the viewing station indicating the identified location of stored pixel data corresponding to the particular medical image.

In some embodiments, rerouting the request for the particular medical image data item includes automatically rerouting, by the PACS, the request to the large image manager component in response to receiving the request for pixel data from the viewing station.

In some embodiments, the method further comprises: defining a plurality of large medical imager types, the plurality of large medical imager types comprising a medical imager configured to: generate a large medical image data item; determining, by the PACS from the received request, that the particular medical image data item was generated by a medical imager corresponding to one of the larger medical imager types; and in response to determining that the particular medical image data item was generated by a medical imager corresponding to one of the larger medical imager types generated by one of the medical imagers, and in response to determining that a particular medical image data item was generated by a medical imager corresponding to one of the large medical imager types, the request is automatically re-routed by the PACS to the large image manager component.

In some embodiments, the large medical image data item comprises a plurality of sub-images of the series of medical images, and the method further comprises: storing image-specific pixel data for each of the sub-images of the plurality of sub-images in the series of medical images, wherein Image-specific pixel data for at least some of the sub-images is stored separately from image-specific pixel data for other parts of the sub-images.

In some embodiments, the received request identifies one of the sub-images of the series of medical images corresponding to the particular medical image; and providing, by the large image manager component, the particular pixel data corresponding to the particular medical image comprises responding to the rerouted request Instead, provide the image-specific pixel data for the requested sub-image.

In some embodiments, providing, by the large image manager component, the specific pixel data corresponding to the specific medical image includes providing the specific pixel data directly to the viewing station in response to the rerouted request.

In some embodiments, providing, by the large image manager component, the specific pixel data corresponding to the specific medical image includes providing the specific pixel data to the viewing station via the PACS in response to the rerouted request.

In some embodiments, the PACS is configured to communicate using the PACS communication protocol; the medical imager is configured to generate large medical image data items in a format different from that defined by the PACS communication protocol; and the large image manager component is configured to generate at least one representative data object in a format defined by the PACS communication protocol.

According to another broad aspect, there is provided a computer program product comprising a non-transitory computer-readable storage medium storing computer-executable instructions for configuring a processor to perform a method of managing medical image data items, wherein the method defined in this article.

According to another broad aspect, a method of collecting medical image data items is provided. The method includes: identifying by a central processing unit a new medical image data file on a remote image storage memory away from the central processing unit; defining a unique file identifier for the new medical image data file by the central processing unit; Determining that the storage of the new medical image data file on the remote image storage memory has been completed to determine that the new medical image data file is a complete data file; the completed data file is copied to the central storage memory communicated with the image management system; the copied The unique file identifier of the data file is stored in the image source database on the central storage memory; identifies the expected set of medical image data files corresponding to the copied data file, the set of expected medical image data files defines the medical image data item and including at least one medical image data file comprising a replicated data file; defining a unique collection identifier for a collection of medical image data files defining a medical image data item, the unique collection identifier identifying the medical image data in the medical image data item a unique file identifier for each of the files; determining that each of the medical image data files of the medical image data item has been copied to the central storage memory; and making the medical image data item available to the image management system at the central storage memory indicator.

In some embodiments, identifying a new medical image data file includes: identifying, by the central processor, a set of image data files stored on the remote image storage memory; The remote file identifier corresponding to the image data file in the file collection; the remote file identifier is compared with the unique file identifier stored in the image source database by the central processing unit; the new medical image data file is compared by the central processing unit One of the image data files identified as having a remote file identifier that does not match any of the unique file identifiers stored in the image source database.

In some embodiments, the method further comprises: monitoring, by the central processor, the collection of image data files stored on the remote image storage memory; determining from the monitoring that additional files are stored on the remote image memory; and determining, by the central processor The remote file identifier serves as the file identifier of the attached file, so that a new medical image data file is identified from the attached file.

In some embodiments, determining that the storage of the new medical image data file on the remote image storage memory has been completed includes: determining the file characteristics of the new medical image data file on the remote image storage memory; a file type, the file type indicating a type of medical image data item defined by the set of medical image data files; determining a complete set of file properties for the determined file type; comparing the file properties with the complete set of file properties; And when the file properties match the complete set of file properties, it is determined that the storage of the new medical image data file on the remote image storage memory has been completed.

In some embodiments, determining that the storage of the new medical image data file on the remote image storage memory has been completed includes: determining the file characteristics of the new medical image data file on the remote image storage memory; An updated file characteristic of the new medical image data file; comparing the file characteristic with the updated file characteristic; and determining storage of the new medical image data file on the remote image storage memory when the file characteristic matches the updated file characteristic Has been completed.

In some embodiments, the method further comprises: when identifying a new medical image data file, assigning the new medical image data file to a new file collection, the new file collection comprising an ordered plurality of medical image data files ; When a new medical image data file is assigned to a new file collection, determine the file characteristics of the new medical image data file on the remote image storage memory; determine that the new medical image data file has arrived in the new file collection to be The copy location indicates that the new medical image data file will be copied to the central storage memory; and when the new medical image data file arrives at the location to be copied, the updated file characteristic of the new medical image data file is determined.

In some embodiments, the medical image data files are associated with a medical imager type; and an expected set of medical image data files is determined from the medical imager type.

In some embodiments, the unique file identifier of the new medical image data file is defined by the central processor independently of the file characteristics of the new medical image data file defined in the remote image storage memory.

In some embodiments, the central processing unit is prevented from modifying files on the remote image storage memory.

In some embodiments, after each of the medical image data files of the medical image data item are copied to the central storage memory, the central processor is configured to ignore deletion of the medical image data of the medical image data item from the remote image storage memory Any medical image data file in the file.

In some embodiments, the central processor is configured to store each of the medical image data files of the medical image data item in their original format on the central storage memory. In some embodiments, the central processor is prevented from modifying medical image data files stored on the central storage memory in their original format. In some embodiments, the central processor generates a digital signature for each of the medical image data files stored in their original format, wherein the digital signature of the particular medical image data file indicates that the particular medical image data file has not been modified.

In some embodiments, the method further comprises: determining, by the central processor for the additional medical image data item, that at least one additional medical image data file in a set of expected additional medical image data files defining the additional medical image data item will not be available for copying from the remote storage memory; and deleting any additional medical image data items of the additional medical image data items that have been copied to the central storage memory.

According to a broad aspect, there is provided an image collection system comprising: a central storage memory having an image storage portion and an image source database, the central storage memory being in communication with an image management system; and a central processing unit coupled to the central storage memory, and coupled to a remote image storage memory remote from the central processing unit and remote from the central storage memory, wherein the central processing unit is configured to: identify a new medical image data file on the remote image storage memory; define a unique file for the new medical image data file Identifier; By determining that the storage of the new medical image data file on the remote image storage memory has been completed, it is determined that the new medical image data file is a complete data file; the completed data file is copied to the image storage part; the copied A unique file identifier for the data file is stored in the image source database; identifying a set of expected medical image data files corresponding to the copied data file, the set of expected medical image data files defining the medical image data item and comprising the copied data at least one medical image data file of files; defining a unique set identifier for a set of medical image data files defining a medical image data item, the unique set identifier identifying each of the medical image data files in the medical image data item A unique file identifier; identifying each of the medical image data files of the medical image data item has been copied to the image storage portion; and generating an indicator that the medical image data item is available to the image management system on the central storage memory.

In some embodiments, the central processor is configured to identify new medical image data files by: identifying a set of image data files stored on the remote image storage memory; remote file identifiers corresponding to image data files in the collection of files; comparing the remote file identifiers with unique file identifiers stored in the image source database; and identifying a new medical image data file as having the remote file identifier remote file identifier that does not match any of the unique file identifiers stored in the image source database.

In some embodiments, the central processor is configured to: monitor a collection of image data files stored on the remote image storage memory; determine from the monitoring that additional files are stored on the remote image storage memory; determine the remote file identifier as A file identifier of the attached file such that the new medical image data file is identified from the attached file.

In some embodiments, the central processor is configured to determine that storage of the new medical image data file on the remote image storage memory has been completed by: determining file characteristics of the new medical image data file on the remote image storage memory; Determining the file type of the new medical image data file, the file type indicating the type of medical image data item defined by the collection of medical image data files; determining a complete set of file characteristics for the determined file type; comparing the file characteristics with the complete and comparing the set of file characteristics with the complete set of file characteristics; and when the file characteristics match the complete set of file characteristics, it is determined that the storage of the new medical image data file on the remote image storage memory has been completed.

In some embodiments, the central processor is configured to determine that storage of the new medical image data file on the remote image storage memory has been completed by: determining file characteristics of the new medical image data file on the remote image storage memory; Subsequently determining updated file characteristics of the new medical image data file on the remote image storage memory; comparing the file characteristics with the updated file characteristics; and determining the new medical image data only if the file characteristics match the updated file characteristics Storing of the file on the remote image storage memory is complete.

In some embodiments, the central processing unit is further configured to: when identifying a new medical image data file, assign the new medical image data file to a new file set, the new file set comprising an ordered plurality of medical image data files Image data files; when new medical image data files are assigned to the new file collection, determining file characteristics of the new medical image data files on the remote image storage memory; determining that the new medical image data files have arrived at the new file collection The position to be copied in indicates that the new medical image data file will be copied to the image storage part; and when the new medical image data file arrives at the position to be copied, the updated file characteristic of the new medical image data file is determined.

In some embodiments, the medical image data files are associated with a medical imager type; and the central processor is configured to determine the expected set of medical image data files from the medical imager type associated with the medical image data files.

In some embodiments, the central processor is configured to define the unique file identifier of the new medical image data file independently of the file characteristics of the new medical image data file defined in the remote image storage memory.

In some embodiments, the central processing unit is prevented from modifying files on the remote image storage memory.

In some embodiments, the central processor is configured to ignore deleting the medical image of the medical image data item from the remote image storage memory once each of the medical image data files of the medical image data item has been copied to the image storage portion Any medical image data file in the data file.

In some embodiments, the central processor is configured to store each of the medical image data files of the medical image data item in their original format on the central storage memory. In some embodiments, the central processor is prevented from modifying medical image data files stored on the central storage memory in their original format. In some embodiments, the central processor generates a digital signature for each of the medical image data files stored in their original format, wherein the digital signature of the particular medical image data file indicates that the particular medical image data file has not been modified.

In some embodiments, the central processor is further configured to: determine for the additional medical image data item that at least one additional medical image data file in the set of expected additional medical image data files defining the additional medical image data item will not be available copying from the remote storage memory; and deleting any additional medical image data items of the additional medical image data items that have been copied to the central storage memory.

According to a broad aspect, there is provided a computer program product comprising a non-transitory computer-readable storage medium storing computer-executable instructions for configuring a processor to perform a method of collecting a medical image data item, wherein the method is herein described definition.

According to a broad aspect, a method of managing medical image data items is provided. The method includes receiving an item of medical image data, the item of medical image data comprising at least one set of medical images, wherein each set of medical images in the at least one set of medical images is from the same medical imaging program, and defining a set of medical images from the medical imaging program sub-images, and each set of medical images has a corresponding resolution and includes at least one sub-image object; defining a data item identifier of the received medical image data item; analyzing the received medical image data item to identify image metadata and image pixel data; generating a plurality of pixel objects for the medical image data item using the image pixel data, wherein each pixel object includes at least a portion of the image pixel data corresponding to one of the sub-images, each sub-image having at least one corresponding pixel object , each portion of the image pixel data is included in one of the pixel objects, and each pixel object also includes an identifying data item identifier corresponding to the pixel object and a pixel object identifier for the sub-image; storing the plurality of pixel objects in In the first storage memory, wherein each pixel object has an address location in the first storage memory; using the image metadata to generate at least one representative object of the medical image data item, the at least one representative object includes a set corresponding to each medical image set A set of representative objects, and each set representative object defines the identification characteristics of the corresponding sub-image of the medical image set, and identifies the first storage memory in which the corresponding at least one pixel object is stored; and stores the at least one representative object in the second storage memory.

In some embodiments, each pixel object corresponds to one of the sub-images and includes all image pixel data from that sub-image.

In some embodiments, the address location of each pixel object is individually addressable in the first storage memory.

In some embodiments, for each pixel object in the plurality of pixel objects, the address location in the first storage memory at which the pixel object is stored is determined independently of the address location of any other pixel object in the other pixel objects .

In some embodiments, each set representative object includes a representative metadata object comprising a metadata defining the identifying characteristics of the corresponding sub-image and identifying the first storage memory in which the corresponding at least one pixel object is stored. data.

In some embodiments, the method further comprises: generating at least one representative object to include at least one overview object of the medical image data item, wherein each overview object includes an overview pixel object and overview object metadata, the overview pixel object derived from the image The selected portion of the pixel data is generated and represents an overview of the set of medical images from a medical imaging program at an overview resolution less than the resolution of the image pixel data in at least one of the medical image sets, and the overview object metadata includes information related to the image pixel data A portion of the image metadata corresponding to the selected portion and a representative object identifier identifying each of the at least one collection representative object; and storing the at least one overview object in a second storage memory.

In some embodiments, generating at least one overview object includes: generating a plurality of overview objects for one of the medical image sets in the medical image data item, wherein each overview object in the plurality of overview objects has a different overview resolution. Overview pixel object.

In some embodiments, the method further includes defining profile object accessibility criteria governing access to each of the profile objects based on the profile object metadata and roles assigned to users attempting to access the profile objects.

In some embodiments: the second storage memory is accessible using a picture archiving and communication system (PACS) that communicates using a defined communication protocol; and each representative object is generated according to the defined communication protocol.

In some embodiments, the received medical image data items are in a format that is not compatible with the defined communication protocol.

In some embodiments, the method further includes generating at least one set of pixel metadata objects, each set of pixel metadata objects corresponding to one of the medical image sets, and each set of pixel metadata objects including a definition corresponding to Identification metadata identifying characteristics of pixel objects of the medical image collection and relational metadata defining spatial relationships between pixel objects corresponding to the medical image collection.

In some embodiments, the method further includes storing at least one set of pixel metadata objects in the first storage memory.

In some embodiments, the first storage memory and the second storage memory are remote from each other and accessible via different image management systems.

According to a broad aspect, there is provided a computer program product comprising a non-transitory computer-readable storage medium storing computer-executable instructions for configuring a processor to perform a method of managing medical image data items, wherein the method is described herein defined in.

According to a broad aspect, there is provided a system for managing medical image data items, comprising: a first storage memory; a second storage memory; and at least one processor coupled to the first storage memory and the second storage memory, Wherein at least one processor is configured to: receive a medical image data item, the medical image data item includes at least one medical image set, wherein each medical image set in the at least one medical image set is from the same medical imaging program and defined from a medical sub-images of an imaging procedure, and each set of medical images has a corresponding resolution and includes at least one sub-image object; defining a data item identifier of a received medical image data item; analyzing the received medical image data item to identify an image metadata and image pixel data; generating a plurality of pixel objects for the medical image data item using the image pixel data, wherein each pixel object includes at least a portion of the image pixel data corresponding to one of the sub-images, each sub-image having at least one corresponding pixel objects, each portion of the image pixel data is included in one of the pixel objects, and each pixel object also includes an identifying data item identifier corresponding to the pixel object and a pixel object identifier for the sub-image; combining the plurality of pixel objects stored in a first storage memory, wherein each pixel object has an address location in the first storage memory; generating at least one representative object of a medical image data item using the image metadata, the at least one representative object comprising set corresponding set representative objects, and each set representative object defines an identification characteristic of a corresponding sub-image of the medical image set, and identifies a first storage memory in which the corresponding at least one pixel object is stored; and the at least one representative object Objects are stored in a second storage memory.

In some embodiments, the at least one processor is configured to generate each pixel object to correspond to one of the sub-images and to include all image pixel data from that sub-image.

In some embodiments, the at least one processor is configured to store each pixel object at an individually addressable address location in the first storage memory.

In some embodiments, at least one processor is configured to determine, for each pixel object of the plurality of pixel objects, the first pixel object at which the pixel object is stored independently of the address location of any other pixel object of the other pixel objects. An address location in storage memory.

In some embodiments, the at least one processor is configured to generate each collection representative object as a representative metadata object comprising identification properties defining the corresponding sub-image and identifying at least one of the corresponding sub-images stored therein. Metadata for the first storage memory of a pixel object.

In some embodiments, the at least one processor is configured to: generate at least one representative object to include at least one overview object of the medical image data item, wherein each overview object includes an overview pixel object and an overview object metadata, the An overview pixel object is generated from a selected portion of the image pixel data and represents an overview of a collection of medical images from a medical imaging program at an overview resolution less than the resolution of the image pixel data in at least one medical image collection, and overview object metadata including a portion of the image metadata corresponding to the selected portion of the image pixel data and representative object identifiers identifying each of the at least one set representative object; and storing the at least one overview object in a second storage memory.

In some embodiments, the at least one processor is configured to generate at least one overview object by: generating a plurality of overview objects for one of the medical image sets in the medical image data item, wherein each of the plurality of overview objects Each overview object has overview pixel objects with different overview resolutions.

In some embodiments, the at least one processor is configured to: define profile object accessibility that controls access to each of the profile objects based on profile object metadata and a role assigned to a user attempting to access the profile object standard.

In some embodiments, the second storage memory is accessible using a picture archiving and communication system (PACS) that communicates using a defined communication protocol; and the at least one processor is configured to The communication protocol generates each representative object.

In some embodiments, the received medical image data items are in a format that is not compatible with the defined communication protocol.

In some embodiments, the at least one processor is configured to: generate at least one set of pixel metadata objects, each set of pixel metadata objects corresponding to one of the sets of medical images, and each set of pixel metadata objects The collection includes identification metadata defining identifying characteristics of pixel objects corresponding to the collection of medical images and relational metadata defining spatial relationships between pixel objects corresponding to the collection of medical images.

In some embodiments, the at least one processor is configured to store at least one set of pixel metadata objects in the first storage memory.

In some embodiments, the first storage memory and the second storage memory are remote from each other and accessible via different image management systems.

These and other aspects and features of various embodiments are described in more detail below.

Description of drawings

For a better understanding of the described embodiments and to show more clearly how they may be practiced, reference will now be made, by way of example, to the accompanying drawings, in which:

FIG. 1 is a block diagram of an image management system according to an example embodiment;

Figure 2 is a block diagram of an example picture archiving and communication system (PACS);

Figure 3 is a block diagram of a large image manager component according to an example embodiment;

FIG. 4 is a flowchart illustrating a method of managing a large medical image according to an example embodiment;

5 is a flowchart illustrating a method of providing pixel data for a large medical image according to an example embodiment;

6 is a flowchart illustrating a method of collecting medical image data items according to an example embodiment;

FIG. 7 is a flowchart illustrating a method of processing medical image data items according to an example embodiment;

FIG. 8 is a flowchart illustrating a method of transmitting medical image data items to an external system according to an example embodiment; and

FIG. 9 is a block diagram illustrating the transfer of large medical image data items between image management systems according to an example embodiment.

The figures described below are presented for purposes of illustration, not limitation, of various exemplary aspects and features of the embodiments described herein. For simplicity and clarity of illustration, elements shown in the figures have not necessarily been drawn to scale. The dimensions of some of the elements may be exaggerated relative to other elements for clarity. It will be understood that, for simplicity and clarity of illustration, where considered appropriate, reference numerals may be repeated among the figures to indicate corresponding or analogous elements or steps.

Detailed ways

Various systems, methods and computer program products are described below to provide examples of embodiments of the claimed subject matter. The embodiments described below do not limit any claimed subject matter, and any claimed subject matter may cover methods or systems other than those described below. Claimed subject matter is not limited to a system or method having all of the features of any one system or method described below, or to features common to multiple or all of the apparatus or methods described below. It is possible that the system or method described below is not an embodiment recited in any claimed subject matter. Any subject matter disclosed in the systems or methods described below that is not claimed in this document may be the subject of another protective instrument (e.g., a continuation patent application), and applicants, inventors, or proprietors do not It is intended to disclaim, disavow or commit to disclosure of any such subject matter by its disclosure in this document.

It will be appreciated that numerous specific details are set forth in order to provide a thorough understanding of the embodiments described herein. However, it will be understood by those of ordinary skill in the art that the embodiments described herein may be practiced without these specific details. In other instances, well-known methods, procedures, and components have not been described in detail so as not to obscure the embodiments described herein. Furthermore, this description should not be taken as limiting the scope of the embodiments described herein.

It should also be noted that the terms "coupled" or "coupled" as used herein can have several different meanings depending on the context in which the terms are used. For example, the terms coupled or coupled may be used to indicate that an element or device can transmit data to and receive data from another element or device electrically, optically or wirelessly.

It should be noted that terms of degree such as "substantially", "about" and "approximately" as used herein mean an amount of reasonable deviation of the modified term such that the end result is not significantly changed. Terms of these degrees may also be construed to include deviations of the modified term if such deviation would not negate the meaning of the term it modifies.

The terms "an embodiment," "an embodiment," "a plurality of embodiments," "the embodiment," "the embodiments," "one or more embodiments," "some embodiments," and "a An embodiment means "one or more (but not all) embodiments(s) of the invention" unless expressly specified otherwise.

Unless expressly specified otherwise, the terms "include", "comprises" and variations thereof mean "including but not limited to". Unless expressly specified otherwise, a list of items does not imply that any or all of the items are mutually exclusive. The terms "a", "an" and "the" mean "one or more" unless expressly specified otherwise.

Further, any recitation herein of numerical ranges by endpoints includes all numbers and fractions subsumed within that range (eg, 1 to 5 includes 1, 1.5, 2, 2.75, 3, 3.90, 4, and 5). It should also be understood that all numbers and fractions thereof are assumed to be modified by the term "about", which means that the amount of the recited figure varies by up to a certain amount if the end result is not significantly changed.

Example embodiments of the systems and methods described herein may be implemented as a combination of hardware or software. In some cases, example embodiments described herein may be implemented at least in part through the use of one or more computer programs that include at least one processing element and data storage elements (including volatile memory, non-volatile volatile memory, storage elements, or any combination thereof) on one or more programmable devices. By their nature, these devices can also have at least one input device (such as a button keyboard, mouse, touch screen, and the like), and at least one output device (such as a display screen, printer, wireless radio, and the like).

It should also be noted that there may be some elements for implementing at least part of one of the embodiments described herein, which may be implemented via software written in a high-level computer programming language such as object-oriented programming. Thus, program code may be written in C, C++ or any other suitable programming language and may include modules or classes as known to those skilled in the art of object-oriented programming. Alternatively, or in addition, some of these elements implemented via software may be written in assembly language, machine language or firmware, as desired. In either case, the language may be a compiled or interpreted language.

At least some of these software programs may be stored on storage media (eg, computer readable media such as but not limited to ROM, magnetic disks, optical disks) or devices readable by general purpose or special purpose programmable devices. The software program code, when read by the programmable device, configures the programmable device to operate in a new, specific and predefined manner in order to perform at least one of the methods described herein.

Furthermore, at least some of the programs associated with the systems and methods of embodiments described herein may be capable of being distributed in a computer program product comprising a computer program carrying computer usable instructions for one or more processors. computer readable media. The media may be provided in a variety of forms, including non-transitory forms such as, but not limited to, one or more magnetic disks, compact disks, tape, chips, and magnetic and electronic storage devices.

Although the use of electronic document management systems is becoming more common in the medical discipline, current image management systems may be ill-equipped to handle the size and volume of medical images generated. Often, image management systems operate independently within a particular department, and medical images generated by different imaging processes or by different imagers in other departments may not be apparent or usable. For example, different image management systems may operate using different file formats or communication protocols.

In some cases, medical image data items may be converted to a standard format or protocol, such as the DICOM standard. While this may allow integration with centralized image management systems such as PACS, there may still be technical challenges in managing large medical image data items. Existing core PACS architectures (and legacy systems) may not be technically adequate for the increases in performance, throughput, and volume that are often necessary to manage large medical image data items (especially In large volumes) required.

The core PACS workflow and storage policies may not be configured to support the network bandwidth and storage capabilities associated with large medical image data items. Because the DICOM standard is primarily concerned with the standardization of data transmission/communication, the protocol is not defined with a focus on issues of storage, retrieval and display performance and feasibility at the forefront of managing large medical image data items.

While updating the core PACS architecture can alleviate some of these issues, it is a long and expensive process. Updating the core PACS architecture typically requires extensive development, verification, validation and regulatory approval. At the same time, additional medical image data items are generated, and the medical image data items continue to grow in size.

For example, consider a tissue slide sample approximately 20 mm x 15 mm in size. The sample can be digitized at a resolution of 0.25 micrometers per pixel (micrometers per pixel or mpp), which typically corresponds to an optical microscope with 400X magnification. The resulting medical image may be approximately 80,000 pixels by 60,000 pixels or 4.8 gigapixels. A corresponding digital medical image generated in 24-bit color may then have a data size of about 15GB.

Medical image data items of even larger data sizes can also be generated by the medical imager. For example, tissue samples up to 50 mm x 25 mm in size can be captured from conventional 1" x 3" tissue slides, and even larger sample sizes can exist on 2" x 3" tissue slides. Medical images can also be digitized at resolutions higher than 0.25mpp—for example, some scanning instruments now support oil-immersion lenses that can magnify up to 100x, yielding a resolution of about 0.1mpp.

For some sample types, tissue samples may be thicker than the depth of field of the objective. As a result, multiple focal planes ("Z-planes") may be captured as sub-images of a medical image generated for a tissue sample. A tissue sample of 50 mm×25 mm can be digitized into a medical image data item with 10 Z planes at 0.1 mpp to obtain a medical image data item including 10 sub-images each with a size of 500000×250000 pixels. Each planar sub-image will be about 125Gp or 375GB of data, and the entire medical image data item will be about 3.75TB of data. Alternatively, multispectral imaging can capture up to 10 spectral bands at a resolution of 16 bits per pixel.

As resolution increases, and the ability to capture more sub-images (eg, additional z-planes) also increases, the data size of medical image data items will continue to increase. Accordingly, medical image management systems may require programs adapted to facilitate the use of these large medical image data items.

Embodiments described herein may provide a centralized system configured to access and manage medical image data items generated by different imagers and in different formats. The system can provide centralized access to medical images that would have previously been accessible only through department specific document management systems. Embodiments described herein may also provide systems and methods capable of identifying, collecting, processing, and managing large numbers of large medical image data items. The described embodiments may alleviate some of the inefficiencies of current systems for managing medical image data items.

Embodiments of the systems, methods, and computer program products described herein generally relate to managing medical image data items. As used herein, the term "medical image data item" refers to a collection of electronic data defining an electronic medical image. A medical image data item is formed from a collection of files comprising one or more electronic files that together include all pixel data and metadata defining a medical image (possibly including multiple sub-images) and associated attributes.

Typically, medical image data items are generated by medical imagers. Individual medical image data items typically correspond to medical imaging procedures, such as digitizing tissue sample slides, performing x-ray or computed tomography scans, or capturing eg endoscopic video. In the example of endoscopic video, the medical image data item may include sub-images corresponding to each frame of the video.

A medical image data item may also include additional metadata defining characteristics or attributes of the medical image, such as patient data or other data. The format of individual medical image data items may vary and may be modified during the various processes described herein.

In particular, embodiments described herein may facilitate the collection, processing and/or management of large medical image data items. For example, all medical image data items generated by one or more managers may be stored in a storage component external to the core workflow of an existing image management system. Representative objects can be generated for each of those medical image data items and then provided to existing image management systems for use in core workflows. Representative objects can be generated with a data size smaller than the medical image data items themselves to facilitate management within existing workflows. For example, representative objects may be limited to a maximum data size of 100MB in some examples. In some examples, representative objects may be limited to a maximum data size of 10MB. Embodiments described herein may provide a centralized system that provides clinicians with access to large items of medical image data from multiple imagers. In some cases, a centralized system may include medical image data items originally generated by multiple imagers having different imaging types and/or medical image data items originally generated in different formats.

Some embodiments described herein may provide image management systems and methods that operate with existing image management systems. For example, embodiments described herein may operate in conjunction with the core components of a PACS system, such as the PACS described in US Patent No. 6,574,629, which is incorporated herein by reference in its entirety.

Although the description may refer to a PACS system for clarity and brevity, it should be understood that embodiments of the systems and methods described herein may be applied to other medical image management systems unless otherwise indicated. Also, although the description may refer to image management systems operating using the DICOM communication protocol, it should be understood that embodiments of the systems and methods described herein may be applied to image management and document management systems using other defined communication protocols and standards.

Embodiments described herein may provide a large image manager or large image manager component (which may be referred to herein as a "LIM"). The LIM can store large medical image data items in a storage memory external to a core component of an existing medical image management system such as a PACS. The large image manager component may provide representative data objects representing large medical image data items stored in the LIM to a medical image management system (eg PACS). These representative data objects can then be incorporated into existing image management workflows. Representative data objects can be generated with limited data sizes to facilitate incorporation into workflows. The data size of the representative data object may be significantly smaller (eg, 100, 1000 or 10000 times) than the data size of the large medical image data item stored in the LIM. When a request for a medical image corresponding to a large medical image data item is received by the PACS, the request may be rerouted to the LIM. The LIM can then handle the re-request and provide the pixel data for the requested image.

In embodiments described herein, existing medical image workflows can be used to manage the life cycle of large medical image data items similar to smaller medical image data items. However, large medical image data items can be stored, moved, and processed externally from the core components that manage the image lifecycle. Thus, the lifecycle of large medical image data items can be managed without introducing delays or bottlenecks due to inefficient transmission of large medical image data items in the core components of the PACS.

In some embodiments, the LIM can be completely external to the PACS. Alternatively, LIM can operate as an extension or plug-in within the PACS system. When LIM operates as an extension or plug-in, it can still operate outside the workflow of the core PACS system (e.g., by storing large medical image data items in separate sections of memory from the corresponding representative objects used in the workflow middle).

The LIM can be configured with a separate file system and messaging system from the PACS. LIM can also use a different communication protocol than PACS for its internal file management and image management processes.

A LIM can acquire medical image data items generated by one or more medical imagers. The medical image data items may include data items corresponding to medical images from imagers, such as digital pathology scanners, endoscopic cameras, X-ray scanners, CT scanners, and the like. In some cases, medical image data items may be generated in a DICOM compatible format, such as DICOM WSI. In other cases, medical image data items may be generated in other formats, such as imager-specific proprietary formats that may not be directly compatible with the DICOM protocol.

In some cases, the LIM can communicate directly with the medical imager. Alternatively, the LIM may collect or acquire medical image data items from an image storage database, imaging archive, and/or image management system associated with the medical imager.

A LIM typically includes one or more processors and LIM memory. Medical image data items collected by the LIM may be stored in the LIM memory. A LIM may provide access to medical image data items through an interface that conforms to a communication protocol used or defined by the core components of a medical image management system such as PACS. For example, the LIM may include a DICOM interface or API configured to communicate with the PACS in a DICOM compliant manner.

In some embodiments described herein, large medical image data items may be stored separately from smaller medical image data items. For example, small medical image data items (e.g., those below a threshold data size) may be stored in a storage memory forming part of a core component of the PACS, while large medical image data items (e.g., equal to or larger than medical image data items of the threshold data size) are stored in a separate storage memory included in the LIM.

The threshold data size may vary due to system configuration and requirements of different individual and organizational users. In some cases, the threshold data size may vary based on the network infrastructure configuration and/or data storage considerations of a particular system implementation.

In other embodiments, all medical image data items may be stored in the LIM. For example, raw medical image data items may be stored in the LIM in an unmodified or unprocessed state to provide a single repository for raw captured medical image data items. This can facilitate regulatory compliance, for example, when medical image data items must be stored in their original format.

LIM can generate representative objects for collected medical image data items. Representative objects may include objects in DICOM format that may be provided to the PACS. PACS can then use these DICOM representation objects in core PACS workflows. For example, a PACS can integrate these DICOM representation objects into its core data lifecycle management and data access workflows. DICOM representative objects can be integrated into the workflow along with small medical image data items stored in the core PACS system.

This could allow the PACS to provide integrated documentation and lifecycle management for medical image data items from different departments and different imaging programs. This may also provide clinicians with a centralized database of available medical image data items, and centralized access to these medical image data items.

LIM can be integrated with PACS to modify the workflow of medical image data items stored in LIM. For example, when a PACS workflow results in requests to provide, deliver, modify, and/or delete medical image data items, these requests can be rerouted to the LIM to process the request. The LIM may then perform the requested action or part of the requested action involving the data stored in the LIM. When a workflow or request involves small medical image data items stored in the PACS, in embodiments where those small medical image data items are stored in the PACS, the request may be processed by the core components of the PACS according to a conventional workflow. Alternatively, in embodiments where all medical image data items are stored in the LIM, they can still be rerouted to the LIM.

The LIM can collect medical image data items in an "as-is" state. That is, the LIM can collect medical image data items in a format defined by the imager for generating those medical image data items. The LIM may also store the collected medical image data items in their original format. These medical image data items may not be modifiable when stored in their "as is" state. Rather, any processing may be limited to reading and/or extracting data stored in raw medical image data items.

The LIM memory may include a large image storage archive where collected medical image data items may be stored in their original format and state. This ensures compliance with regulatory requirements and provides data fidelity. The LIM may also generate a digital signature for the collected medical image data items, the digital signature indicating that the stored medical image data items have not been modified from their originally collected state. This ensures that medical image data items are stored in accordance with regulatory obligations.

Although the description may describe the operation of the LIM in the context of operation of the PACS, it should be understood that the various features and components of the LIM may operate independently of the PACS unless otherwise indicated. Similarly, although various systems and methods may be described herein with reference to PACS and/or LIM, it should be understood that these systems and methods may be implemented in other systems unless otherwise indicated. For example, the systems and methods described herein for collecting medical image data items can be implemented with or without PACS and/or LIM. Similarly, the systems and methods described herein for processing medical image data items can be implemented with or without PACS and/or LIM.

Referring now to FIG. 1 , therein is shown a block diagram of an image management system 100 according to an example embodiment. In the example shown in FIG. 1 , system 100 includes imager manager 102 communicatively coupled to imager 110 and to at least one information storage system 112 . Image manager 102 includes large image management component (LIM) 104 coupled to picture and archive communication system (PACS) 106 .

In general, system 100 typically includes a plurality of computers that may be connected via a data communications network. The data communication network may include both local and extensive communication networks, and may be connected to the Internet.

Typically, the connection between the data communication network and the Internet can be made via a firewall server (not shown). In some cases, there may be multiple links or firewalls or both between the data communications network and the Internet. Some organizations may operate multiple networks or virtual networks, which may be interconnected or isolated. These have been omitted for ease of illustration, however it will be understood that the teachings herein may be applied to such systems. The network may consist of one or more computer network technologies such as IEEE 802.3 (Ethernet), IEEE 802.11 and similar technologies.

Computers and computing devices such as imager 110 , information storage system 112 , image manager 102 (ie, LIM 104 and PACS 106 ), and viewer 108 may be connected to a data communications network, or portions thereof, via suitable network interfaces. In some cases, one or more of the computing devices, such as viewer 108, may be connected to a data communications network via the Internet.

Examples of computers that may be used to implement components of system 100 such as LIM 104, PACS 106, and viewer 108 may include desktop or laptop computers. These computers may be connected to the data communications network via a wired Ethernet connection or a wireless connection. Imager manager 102 (including large image manager component 104 and PACS 106 ) can also connect to a data communication network via the Internet.

Viewer 108 has a processor, volatile and non-volatile memory, at least one network interface, and may include input devices such as a keyboard and a touchpad, and output devices such as a display and speakers, and Various other input/output devices are understood.

Image manager 102 has at least one processor, volatile memory and non-volatile storage memory, at least one network interface, and may include input devices such as a keyboard and trackpad, and output devices such as a display and speakers , and various other input/output devices as will be understood. In some cases, image manager 102 may include a plurality of linked server computers accessible through a data communications network (eg, via the Internet). Accordingly, image manager 102 may not directly require input devices and/or output devices.

The large image manager component 104 also typically includes at least one processor, volatile and non-volatile storage memory, and at least one network interface. The large image manager component 104 can be implemented as a combination of hardware and software, and can include a software program that can be stored on a storage medium or device that can be read by a general purpose or special purpose programmable device. LIM 104 may also be distributed among multiple instances of processor and/or storage memory.

PACS 106 also typically includes at least one processor, volatile and non-volatile memory, and at least one network interface. PACS 106 may be implemented as a combination of hardware and software, and may include a software program that may be stored on a storage medium or device readable by a general-purpose or special-purpose programmable device. PACS 106 may also be distributed among multiple instances of processor and/or storage memory.

In some examples, there may be overlap between processors and/or memory used by large image manager component 104 and PACS 106 . In other cases, each of large image manager component 104 and PACS 106 may use entirely separate processors and memory. Typically, however, the large image manager component 104 operates separate file systems and messaging systems from the PACS 106 and has separate data storage memory. Large image manager component 104 and PACS 106 may communicate using a communication interface, such as DICOM interface 350 shown in FIG. 3 and discussed below.

Some of the computers in system 100, such as viewer 108, may be implemented by mobile devices such as smartphones or tablet computers. Mobile devices can generally include a wide variety of "smart" devices capable of data communication. Typically, a mobile device has a processor, volatile and non-volatile memory, at least one network interface, and input/output devices. Mobile devices are generally portable and may sometimes be connected to a data communications network or part thereof. For example, a mobile device can remotely connect to a data communication network using a virtual private network (VPN).

Typically, the processor used by the computer components of system 100 may be a computer processor, such as a general-purpose microprocessor. In some other cases, a processor may include a field programmable gate array, an application specific integrated circuit, a microcontroller, or other suitable computer processor.

The processor can be coupled to the memory via a computer data bus. Memory can include both volatile and nonvolatile memory. Non-volatile memory stores a computer program consisting of computer-executable instructions, which can be loaded into volatile memory for execution by a processor as needed. Those skilled in the art will appreciate that reference herein to a component such as LIM 104 or PACS 106 as performing a function or acting in a particular manner means that one or more processors are executing instructions (e.g., software programs) stored in memory , and may transmit or receive input and output via one or more interfaces. The memory may also store data that is input to or output from the processor during execution of the computer-executable instructions.

The processor may also be coupled to one or more displays suitable for outputting information and data as required by various computer programs. In particular, the display may display a graphical user interface (GUI) to a user interacting with image manager 102 . For example, viewer 108 may include a display that provides a GUI for a user to interact with imager manager 102 via viewer 108 . In some cases, such as where LIM 104 or PACS 106 are configured to operate autonomously, a display may be omitted from LIM 104 or PACS 106 . In such cases, LIM 104 or PACS 106 may be configurable using a computer such as viewer 108 connected to image manager 102 . The various components of system 100, such as LIM 104, PACS 106, and viewer 108, may each execute an operating system, such as Microsoft Windows ^™ , GNU/Linux, or other suitable operating system.

Each of the computers and computing devices used to implement the components of system 100 may sometimes be connected via the Internet to external computers or servers. For example, LIM 104 and/or PACS 106 may be connected to an external image storage archive that may be stored remotely or "in the cloud."

As used herein, the term "software application" or "application" refers to computer-executable instructions, particularly computer-executable instructions stored on a non-transitory medium such as non-volatile memory and executed by a computer processor. instruction. A computer processor may receive input and transmit output to any of a variety of input or output devices to which it is coupled in executing instructions.

The image manager 102 may be used to provide various functions and operations related to the storage and management of medical image data items. In particular, the image manager 102 shown in FIG. 1 can facilitate managing large medical image data items. The image manager 102 may also facilitate the management of medical image data items generated by different imagers and by different imaging file formats.

In some cases, image manager 102 may be used to implement various processes for managing medical image data items, such as methods 400 and 500 described herein below. In some cases, image manager 102 may be used to implement various processes for collecting medical image data items, such as method 600 described herein below. In some cases, image manager 102 may be used to implement various processes for processing medical image data items, such as method 700 described herein below. In some cases, image manager 102 may be used to implement various processes for communicating medical image data items, such as method 800 described herein below.

Image manager 102 generally includes a large image manager component or LIM 104 and PACS 106 . LIM 104 and PACS 106 may operate in conjunction to collect, store, retrieve, process, provide access to, and display medical image data items.

An example of a PACS 206 that may be used to implement PACS 106 is shown in FIG. 2 and described in further detail below. Typically, PACS 106 includes a number of core components implemented as a combination of hardware and software. These core components provide functionality such as archiving, reviewing, and displaying medical images. In some cases, viewer 108 may be considered a core component of PACS 106 . For example, the core components of PACS 106 may generally correspond to those of the PACS described in US Patent No. 6,574,629.

PACS 106 includes a workflow activation component 114 that defines workflows for managing data stored in PACS 106 . A workflow defined by workflow activation component 114 may be initiated in response to user input, such as a request to review a medical image or to transfer a medical image data item to an external system. Workflows defined by the workflow activation component 114 can also be automatically started in response to system triggers, such as a period of time that elapses after certain actions have taken place.

Workflows may include lifecycle management workflows, which may define where and how different medical image data items are stored (e.g., how and when medical image data items should be archived, how medical image data items should how and when it was modified and/or deleted, etc.). Workflows can include image access workflows that can define when and how medical image data items are provided to viewer 108 for review (e.g., how and when medical image data items should be retrieved for access or review , how and when medical image data items should be prefetched for anticipated review, etc.). Workflows may include image transfer workflows that may define when and how medical image data items are transferred between PACS 106 and external image management systems. The workflows can include image collection workflows that can define when and how medical image data items are collected from external image management systems. The workflows may include image processing workflows that may define when and how medical image data items are processed by the imager 102 . For example, an image processing workflow in workflow activation component 114 can trigger a corresponding image processing workflow in LIM 104 . Workflows can also include image reconciliation workflows that can define when and how medical image data items are updated with external systems, such as laboratory information system 112 .

PACS 106 may include image storage memory that may be used to store medical image data items and associated information. The image storage memory can store configuration files for the PACS 106 that can be used by the workflow activation component 114 to determine medical image data items, data collection or delivery rules and destinations, data related to external systems in communication with the PACS 106, and Relationships among other data.

Typically, PACS 106 operates using a defined communication protocol (DICOM in this case). Data stored by PACS 106 is stored and managed according to the DICOM standard protocol. For example, workflows defined by workflow activation component 114 operate according to the DICOM standard.

PACS 106 may also include external interfaces or gateways to interface with external components such as LIM 104 , viewer 108 , imager 110 , and/or information storage system 112 . The external interface can be configured to receive and transmit data and messages according to defined communication protocols (such as DICOM, HL7, FHIR, etc.).

An example embodiment of a LIM 304 that may be used to implement LIM 104 is shown in FIG. 3 and described in further detail below. Typically, LIM 104 includes a combination of hardware and software components that can be used to manage large items of medical image data, eg, as described below with reference to FIG. 3 . LIM 104 may also include an internal workflow activation component similar to workflow activation component 114 of PACS 106 . The workflow activation component of LIM 104 may define workflows specific to LIM 104, such as workflows related to processing medical image data items and/or generating representative objects. In some cases, the workflow defined by the workflow activation component 114 of the PACS 106 may be a more general system/enterprise-wide workflow, while the LIM-specific workflow may be more focused on the storage and processing of medical image data items . LIM 104 workflows can also be triggered by PACS 106 workflows.

LIM 104 may include external communication interfaces to interface with external components such as PACS 106 , viewer 108 , imager 110 , and/or information storage system 112 . Interfaces in LIM 104 may include various APIs configured to communicate using different communication protocols and in different data formats to allow LIM 104 to interface with various external systems. For example, LIM 104 may include a DICOM interface to interface with PACS 106 according to the protocol used by PACS 106 .

In some embodiments, LIM 104 and PACS 106 may operate in combination or in an interdependent manner to provide imager manager 102 . LIM 104 may store medical image data items and generate representative objects from the stored medical image data items. The representative objects can then be used in the image management workflow of the PACS 106 . The PACS 106 may then request full medical image data items or full-size pixel data from those medical image data items only upon user request.

In general, LIM 104 may store all medical image data items to facilitate data management by removing large medical image data items from PACS 106's storage and management workflow. Likewise, LIM 104 may store medical image data items generated in different formats that may not be compatible with PACS 106 workflow. LIM 104 can then generate representative objects as DICOM files that can be managed by core components of PACS 106 .

LIM 104 can generate representative objects with reduced or limited data sizes. In some cases, LIM 104 may generate representative overview objects with a maximum data size of 10MB, which may be transferred to PACS 106 and used in PACS 106 management workflows. In some cases, LIM 104 can generate a representative metadata object with a maximum data size of 5MB or even 1MB, which can then be sent to PACS 106 and used in the management workflow. Overview objects can be larger because they can include overview pixel data that can be reviewed by the user, while representative metadata objects can only include metadata used by the administrative workflow and possibly reviewed by the user. The maximum data size of an overview object and/or representative metadata object may vary according to a particular system implementation.

Alternatively, LIM 104 may manage and store large medical image data items while PACS 106 stores small medical image data items. The distinction between small medical image data items (i.e., those stored in PACS 106) and large medical image data items (i.e., those stored in LIM 104) can generally be based on the data size or expected size of the medical image data items being stored. to determine the size of the data. For example, a threshold data size may be defined by the image manager 102 to distinguish small medical image data items from large medical image data items. A medical image data item larger than or equal to a threshold data size may be determined as a large medical image data item.

The threshold data size may depend on various factors related to the particular implementation of system 100 . For example, the threshold data size may be defined as 10 GB, 25 GB, 50 GB, or 100 GB in different implementation examples.

In some cases, the expected data size of the stored medical image data items may be used to distinguish small medical image data items from large medical image data items. In some cases, the type of medical image data item being stored (e.g., X-ray, endoscopic video, CT scan) or the imager that generated the particular medical image data item may be used to determine where to store the medical image data item LIM 104 is also in PACS 106 . For example, imagers or image types that tend to generate medical images with larger data sizes may be assigned to LIM 104 while other medical image data items are assigned to PACS 106 . Also, the threshold expected data size (and corresponding image type or imager) can vary by implementation.

LIM 104 may perform various operations related to collection, storage and processing of medical image data items. PACS 106 can store representative objects and can perform various operations related to the collection and processing of those representative objects. In some cases, PACS 106 may also store and perform operations related to the collection and processing of small medical image data items, such as those already stored in PACS 106 prior to the addition of LIM 104 . PACS 106 can also manage lifecycle and access to both small and large medical image data items. This can facilitate integration of LIM 104 into existing clinical workflows and clinical procedures.

Image manager 102 may be communicatively coupled to one or more viewers 108 . Viewer 108 may be a computing device operated by a user, such as a clinician, to request, review, and/or analyze medical images, among other functions. A user may interact with image manager 102 through viewer 108 .

Large image manager component 104 can facilitate interaction with PACS 106 by removing large medical image data items from PACS 106's core workflow. This can reduce bandwidth requirements on core PACS 106 components and, in turn, facilitate user interaction with PACS 106 . Meanwhile, requests for large medical image data items transmitted to PACS 106 may be rerouted to LIM 104 to provide the requested medical images to viewer 108 .

From a clinician's perspective, interacting with the PACS 106 to request and review images may appear the same. However, clinicians may be provided with access to an increased number and type of medical image data items. Likewise, clinicians may experience reduced latency when interacting with image manager 102 because the throughput required by PACS 106 to perform its image management operations may be reduced.

As shown by data transmissions 116, 122, and 126 in FIG. 1, LIM 104 and PACS 106 may operate in bi-directional communication to manage medical image data items. For example, LIM 104 may transmit a metadata message 116 to PACS 106 indicating that a new medical image data item has been stored in LIM 104 and is available to be accessed. The metadata message 116 or another message sent by the LIM 104 may also include one or more representative objects corresponding to the new medical image data item. The representative objects can then be used by PACS 106 in image management workflows.

PACS 106 may determine that one of the items of medical image data stored on LIM 104 should be modified and/or updated and/or archived and/or deleted and/or prefetched and/or processed, etc. PACS 106 may then transmit management messages 126 to LIM 104 indicating the appropriate lifecycle actions. LIM 104 may then perform corresponding lifecycle actions related to the medical image data item identified by message 126 .

Image manager 102 may also communicate with one or more viewers 108 . Viewer 108 may correspond to a review station operable to receive and display medical images and sub-images. Viewer 108 and PACS 106 may exchange viewer messages 118 related to medical image data items stored in PACS 106 and/or LIM 104 . Viewer 108 may be configured to communicate using the DICOM protocol used by PACS 106 .

In some cases, viewer 108 may provide request 120 for medical images to PACS 106 . For example, a user of viewer 108 may select medical images to request based on data received in viewer message 118 . If PACS 106 determines that the request corresponds to a representative object (or in some cases a medical image data item) stored in PACS 106, PACS 106 may directly provide the requested pixel data. However, if PACS 106 determines that the requested medical image corresponds to one of the medical image data items stored in LIM 104 , the request may be rerouted to LIM 104 .

In some cases, a rerouted request 122 may be automatically sent by PACS 106 to LIM 104 in response to request 120 . Alternatively, PACS 106 may respond to viewer 108 with an indication that the request needs to be rerouted. Viewer 108 may then transmit rerouted request 122 to LIM 104 .

In response to receiving rerouted request 122, LIM 104 may identify pixel data for the requested medical image. LIM 104 may then transmit an image response message 124 to viewer 108 including the requested pixel data. Image response message 124 may be sent directly to viewer 108 or may be routed through PACS 106 . Viewer 108 may then display the pixel data to the user.

Image manager 102 may be communicatively coupled to one or more imagers 110 . Imager 110 may be configured to generate digital medical images in the form of medical image data items. Examples of imager 110 include digital pathology scanners, endoscopic cameras, X-ray scanners, CT scanners, ultrasound scanners, magnetic resonance imagers, and the like.

In some cases, image manager 102 may not be directly connected to imager 110 . For example, image manager 102 may retrieve medical image data items from an image storage component (ie, storage memory for storing medical image data items) or an archive, which in turn receives medical image data items from imager 110 . The image manager 102 may also receive medical image data items from external image management systems, such as those associated with different healthcare facilities.

Information storage system 112 may store additional data associated with medical image data items generated by imager 110 or stored by image manager 102 . Examples of the information storage system 112 include a Hospital Information System (HIS), a Laboratory/Department Information System (LIS), and/or a Radiology Information System (RIS).

Image manager 102 may communicate with information storage system 112 to retrieve additional metadata related to medical image data items stored by LIM 104 and/or PACS 106 . For example, image manager 102 may communicate with information storage system 112 to retrieve patient data associated with a particular medical image data item. The patient data may then be stored as part of the image metadata of the medical image data item.

Referring now to FIG. 2, there is shown an example of an image management system 200 that includes a PACS 206, such as the LIM 104 shown in FIG. 1, without a large image manager component. An example of a picture archiving and communication system that may be used to implement PACS 206 and its core components is described in detail in US Patent No. 6,574,629.

Typically, PACS 206 is implemented as a combination of hardware and software components, including a processor, volatile memory, non-volatile memory, and communication interfaces. The communication interface may be one or more data network interfaces, such as IEEE 802.3 or IEEE 802.11 interfaces, for communicating over a network.

As shown in FIG. 2 , PACS 206 is communicatively coupled to viewer 208 , information storage system 212 and image archive 230 . PACS 206 may be directly linked to any one or more of viewer 208, information storage system 212, and image archive 230, eg, via a Universal Serial Bus, Bluetooth ^™ , or Ethernet connection. Alternatively, PACS 206 may be linked to any one or more of viewer 208, information storage system 212, and image archive 230 via a data communications network or, in some cases, the Internet.

Typically, PACS 206 operates using a defined communication protocol (DICOM in this example). Data stored by PACS 106 is communicated and managed according to the DICOM standard protocol. For example, workflows defined by workflow activation component 214 operate according to the DICOM standard.

PACS 206 may also include external interfaces or gateways to interface with external components such as viewer 208 , information storage system 212 , and image archive 230 . The external interface can be configured to receive and transmit data and messages according to defined communication protocols (such as DICOM, HL7, FHIR, etc.).

PACS 206 includes workflow activation component 214 and database 232 . Database 232 may be stored in non-voltage storage memory of PACS 206 . In some example embodiments, database 232 is a relational database. In other embodiments, database 232 may be a non-relational database, such as a key-value database, a NoSQL database, or the like.

Database 232 may be used to store medical image data items. The database may also store other data related to medical image data items and management of medical image data items. For example, database 232 may also store system configuration files or workflow settings that define workflows used by workflow activation component 214 . Workflow activation component 214 can be generally similar to workflow activation component 114 . However, workflow activation component 214 may also include workflows related to storage of medical image data items for PACS 206 for implementation without a LIM for storing medical image data items.

Viewer 208 and information storage system 212 may generally correspond to viewer 108 and information storage system 112 shown in FIG. 1 .

Image archive 230 may include storage memory. Image archive 230 may provide long-term persistent storage of medical image data items. Although shown separately from database 232 , image archive 230 may be stored in database 232 in some cases. Alternatively, image archive 230 may be located remotely from PACS 206 .

In the image management system 200 of FIG. 2 , the PACS 206 stores and manages all medical image data items accessible via the PACS 206 . PACS 206 may not be able to access or manage medical image data items generated in a format that is not compatible with the defined communication protocol used by PACS 206 . As a result, a user accessing the PACS 206 through the viewer 208 may only be shown medical image data items generated in the DICOM format.

Also, PACS 206 manages all medical image data items regardless of data size. Regardless of the size of the medical image data item, the workflow defined by the workflow activation component 214 can operate the same workflow. Therefore, determining whether to process a medical image data item or to modify or move a stored medical image data item may not take into account the data size of the medical image data item involved.

Thus, when PACS 206 workflow indicates that a medical image data item is to be transferred from database 232 to image archive 230, or vice versa, this may fully or partially occupy the processor and memory of PACS 206 for the duration of the data transfer. This can cause significant delays or bottlenecks when large medical image data items (sometimes as large as 100 gigabytes or even 10 or 100 terabytes) are being passed around. Similarly, various other operations on large image data items can occupy a large amount of bandwidth available to PACS 206 .

In contrast, in the image management system 100 shown in FIG. 1 , medical image data items are stored in the LIM 104 . Smaller representative objects corresponding to these medical image data items are provided to PACS 106 to be incorporated into the workflow. Thus, the processing and management of large medical image data items can be removed from the PACS 206 with little or no modification to the PACS workflow. This can reduce the stress on the PACS 206 as it manages the life cycle of both small and large medical image data items.

Referring now to FIG. 3 , a block diagram of a large image manager component 304 is shown, according to an example embodiment. Large image manager component 304 is an example of a large image manager component that may be used in image manager 102 as LIM 104 .

LIM 304 typically includes at least one processor and volatile and non-voltage memory. The processor of LIM 304 may be configured to perform various functions related to large medical image data items in an image management system. These functions may be functions that would otherwise be performed by the PACS 106 but were removed from the PACS 106 to reduce stress, and/or may be inefficiently performed because of communication protocols used by the PACS 106 .

As shown in FIG. 3 , LIM 304 may maintain its own file system 340 and messaging system 342 . File system 340 and messaging system 342 may be maintained independently of the file system of PACS 206 .

As noted above, PACS 106/206 may operate according to the DICOM communication protocol. However, the DICOM communication protocol may be inefficiently structured for storing and managing large files. Accordingly, file system 340 and messaging system 342 may operate according to alternative communication protocols.

File system 340 and messaging system 342 of LIM 104 may be configured to handle greater data throughput as compared to PACS 106/206. For example, file system 340 and messaging system 342 of LIM 104 may be configured to have a data throughput of 10 TB or more per day. In contrast, a traditional PACS 206 may be configured to have a throughput of only 10 TB per year. File system 340 and messaging system 342 of LIM 104 may be configured to process medical image data items.

LIM 304 also includes database 352 . Database 352 may be stored in non-volatile storage memory of LIM 304 . Database 352 may be used to store large medical image data items. In some cases, the non-volatile storage memory of LIM 304 may store pixel objects generated from large medical image data items. In some cases, the non-volatile storage memory of LIM 304 may store pixel objects generated from large medical image data items.

LIM 304 may also include a collection component or collector 344 . Collector 344 may communicate with external storage components, such as imager 110 and/or remote storage, to collect medical image data items for storage in database 352 . In some cases, collector 344 may also communicate with information storage system 112 to reconcile additional information with medical image data items stored in database 352 .

In general, collector 344 can be configured to collect images from external image storage components in various ways. For example, collector 344 may replicate medical image data items stored on remote memory. Collector 344 may be implemented as a collector application stored on memory of LIM 304 . An example of an image collection process 600 that may be implemented by collector 344 is shown in FIG. 6 and will be discussed in further detail.

LIM 304 may also include a parsing component or parser 346 . Parser 346 may analyze medical image data items stored in database 352 to identify image metadata and image pixel data. Parser 346 may be implemented as a parsing application stored on memory of LIM 304 .

LIM 304 may receive or collect medical image data items in a variety of different formats from external image storage components. Examples of received data formats may include proprietary formats; proprietary formats based on open source formats such as TIFF, JPEG, BMP, and medical image data items in DICOM format.

Parser 346 may identify the format of the received medical image data item. Based on the format of the received medical image data item, parser 346 may identify the medical image data file and the portion of the medical image data file storing image metadata and image pixel data, respectively. The parser 346 may then extract image metadata and image pixel data from the analyzed medical image data item. Parser 346 may also store image pixel data to facilitate retrieval of requests received from PACS 106 or viewer 108 .

The parser 346 may read and extract metadata stored in the received medical image data item. Parser 346 may store the extracted metadata in a cache storage component to facilitate generation of representative objects. The extracted metadata may then be used, for example, by converter 348 to generate a representative object in a DICOM compatible format. Parser 346 may store the extracted metadata in a cache storage component to facilitate generation of representative objects.

Parser 346 may also process pixel data and metadata of received medical image data items. The parser 346 may generate additional image metadata based on processing of the received medical image data item. This additional metadata can then be used when generating representative objects. For example, parser 346 may analyze pixel data corresponding to one or more sub-images of a medical image data item to identify sub-image metadata that may be included as part of additional image metadata.

LIM 304 may also include conversion component or converter 348 . Converter 348 may generate various objects corresponding to medical image data items received and/or stored by LIM 304 . For example, converter 348 may use extracted image metadata and/or extracted image pixel data from a parser to generate objects corresponding to medical image data items received and/or stored by LIM 304 . Converter 348 may be implemented as a conversion application stored on memory of LIM 304 . An example of an image conversion process 700 that may be implemented by converter 348 is shown in FIG. 7 and will be discussed in further detail.

In general, the converter 348 may generate pixel objects corresponding to medical images and/or sub-images of medical image data items. Pixel objects may be generated from image pixel data extracted by parser 346 . In some cases, converter 348 may also generate pixel metadata objects that define properties of pixel objects and spatial relationships between pixel objects.

The converter 348 may generate representative objects corresponding to medical images and/or sub-images of medical image data items. Some of the representative objects may be provided to PACS 106 for use as part of PACS 106 workflow. Converter 348 can generate some of the representative objects in a DICOM compatible format to allow easy integration with PACS 106 workflows. Examples of representative objects may include representative metadata objects and overview objects. Transformer 348 may also generate representative objects with a limited maximum data size to minimize the size of data objects incorporated into the PACS 106 workflow.

LIM 304 may also include DICOM interface 350 . A DICOM interface may provide an interface or API between LIM 304 and PACS 106 . DICOM interface 350 may be configured to communicate with PACS 106 to transmit and receive DICOM compliant data, such as messages 116 , 122 and 126 shown in FIG. 1 . DICOM interface 350 may be used to transfer DICOM-compliant representative objects to PACS 106 .

DICOM interface 350 may also communicate with viewer 108 configured to operate according to the DICOM protocol. DICOM interface 350 may receive rerouted request 122 from PACS 106 and/or viewer 108 . DICOM interface 350 may then transmit the pixel data or pixel objects to viewer 108 in a format that is DICOM compatible or otherwise usable by viewer 108 .

The database 352 may also store information related to large medical image data items, such as unique file identifiers and unique collection identifiers corresponding to medical image data files and sets of medical image data files, respectively. Database 352 may also include an index or address book that associates unique file identifiers and/or unique set identifiers with memory addresses of corresponding medical image data files or sets of medical image data files. Database 352 may also indicate whether a particular medical image data item is available for access by viewer 108 and/or whether a medical image data item is available for further processing, eg, by parser 346 .

In some cases, non-volatile memory (eg, database 352 ) of LIM 304 may include both a long-term storage portion and a short-term access portion or cache portion. The long term storage portion may provide permanent or virtually permanent storage for medical image data items. In some cases, a long-term storage component may be located remotely from the processor of LIM 304, such as as off-site or cloud storage. This may allow for flexibility and scaling as additional medical image data items (and thus increased amounts of data) are stored in LIM 304 .

A cache storage device may be used to store medical image data items and/or medical image data files in a more easily accessible manner for communication with PACS 106 and/or viewer 108 . For example, a cache storage device may be used to store objects derived from medical image data items or individual files from medical image data items. The cache storage device may store medical image data files or objects that have been requested (eg, by viewer 108 and/or external systems) or are expected to be requested (eg, as a result of prefetching messages from PACS 106 ). Data stored in the cache storage device may be stored in a format compatible with the system that requests or expects to request the object or file. This can provide a faster response to requests from viewer 108 or PACS 106 .

A cache storage device may have a reduced storage capacity as compared to long-term storage components. Accordingly, objects or files stored in a cache storage device may be stored for a limited period of time (or until additional capacity is required). In some cases, objects or files may be stored in cache storage in response to requests or prefetch messages.

In some cases, medical image data items received by LIM 304 may be stored in their original received or "as is" format. This ensures data fidelity for storing medical image data items. In some cases, there may also be regulatory requirements requiring medical image data items to be stored in their original format. Raw medical image data items may be stored in long-term storage.

LIM 304 can also process medical image data items to generate pixel objects and other representative objects. In some cases, LIM 304 may store these pixel objects and/or representative objects in addition to raw medical image data items. This may allow individual medical images or sub-images (and/or associated metadata) to be readily provided in response to requests from PACS 106 and/or viewer 108 .

Alternatively, pixel objects and/or representative objects may be generated in response to a request from viewer 108 (or an anticipated request/prefetch instruction from PACS 106). The pixel objects and/or representative objects may then be temporarily stored, for example, for a period of time during which the pixel objects and/or representative objects are expected to be requested.

In some cases, cache storage may be used to store pixel objects and/or representation objects. In this manner, the cache storage device may be used to store files including data expected to be requested by PACS 106 and/or viewer 108 . The cache storage device may then omit other portions of the medical image data item that are less likely to be requested or otherwise stored in PACS 106 (such as by representative objects that have been transmitted to PACS 106 ).

LIM 304 may also include an internal workflow activation component similar to workflow activation component 114 of PACS 106 . A workflow activation component of LIM 304 may define workflows specific to LIM 304 , such as workflows related to processing medical image data items and/or generating representative objects. LIM 304 may also include cache management workflows that define how and when objects are stored in cache storage and how and when objects are removed or deleted from cache storage.

Workflows defined by LIM 304 can be triggered by workflows defined by PACS 106/206. For example, PACS 106 may define a cache management workflow that triggers a corresponding cache management workflow of LIM 304 . PACS 106 may transmit prefetch messages to LIM 304 to trigger a corresponding cache management workflow.

Referring now to FIG. 4 , therein is shown a flowchart of a method 400 for managing medical images according to an example embodiment. In some cases, method 400 may be implemented as an image management application on an image management system, such as image manager 102 .

At 410, one or more medical image data items are identified. Medical image data items may be identified from a medical imager, such as the imager 110 shown in FIG. 1 . Medical image data items may also be identified from other storage locations, such as those associated with the imager 110 or with other external systems. For example, medical image data items may be identified or received from an external image management system, such as an image management system associated with an external healthcare facility. Medical image data items may also be identified from an image archive, such as image archive 230 shown in FIG. 2 .

Image manager 102 may be coupled to multiple image storage memory locations. Image manager 102 may identify medical image data items at storage memory locations. Image manager 102 may transfer or copy the identified medical image data items to a local or central storage memory associated with image manager 102 . An example process 600 for identifying and collecting medical image data items is described in further detail below with reference to FIG. 6 .

The process of identifying medical image data items may be implemented by either or both of LIM 104 and PACS 106 . In some cases, identifying medical image data items may be incorporated into a general process for identifying and collecting medical image data items. The image manager 102 may determine that one or more of the identified medical image data items are large medical image data items.

In some cases, large items of medical image data may be identified from the aforementioned storage components or imagers. The identified medical image data item may be compared to large image criteria to determine whether the identified medical image data item should be considered a large image data item. For example, a threshold data size may be used to determine whether an identified medical image data item is a large image data item. Alternatively, the image type or data item type may be used to distinguish large medical image data items from small medical image data items (eg, based on the type of imager or imaging process/format used to generate the image data item).

The large medical image data item may then be stored (eg, copied) in LIM 104 . For example, large medical image data items may be stored in database 352 of LIM 304 . In some cases, other identified medical image data items (ie, small medical image data items) may be stored in PACS 106 . In other cases, however, all medical image data items may be stored in LIM 104 to provide a comprehensive image storage memory.

In some cases, medical image data items may be stored in raw database 352 . Alternatively, medical image data items may be processed prior to storage. Alternatively, both processed and unprocessed versions of medical image data items may be stored in LIM 304 .

At 420, pixel data can be identified in the medical image data item. After the raw medical image data item is stored on the LIM 104/304, pixel data may be identified in the medical image data item. Alternatively, pixel data may be identified in medical image data items prior to storage on LIM 104 . This may allow LIM 104 to extract pixel data from medical image data items prior to storage.

The pixel data may be identified by the image manager 102, which identifies a medical image data file in a medical image data item, the medical image data file comprising the pixel data of the medical image and the sub-images of the image data item. In some cases, a subset of medical image data files may be identified as pixel files that include pixel data.

In some cases, image manager 102 may identify a portion of a medical image data file that includes pixel data for a medical image data item. For example, some of the medical image data files may include both image metadata and image pixel data related to the medical image and/or sub-images of the medical image. A pixel data portion of a medical image data file comprising image pixel data may be identified as defining pixel data.

Pixel data may be identified based at least in part on the format of the medical image data item. For example, the format of the medical image data item may indicate that one or more of the medical image data files is an index file comprising metadata relating to the set of medical image data files in the medical image data item. Image manager 102 may parse the index file to identify medical image data files that include pixel data.

The image manager 102 may also determine the portion of the medical image data file that includes pixel data from the file structure definition of the particular format for the medical image data item. For example, the format of a medical image data item may specify how data is arranged within a medical image data file. A format may define a specific metadata section and a specific pixel section. Pixel data can then be identified in a specific pixel portion of the medical image data file. In some cases, an index file may additionally or alternatively identify specific pixel portions of one or more medical image data files.

At 430 , the pixel data may be stored in storage memory of the large imager manager 104 . In some cases, storing pixel data may refer to storage of the medical image data item itself, including the pixel data identified at 420 . For example, a medical image data item may be stored in its original format and state (or at least the format in which it was received and/or collected by the image manager 102).

Additionally or alternatively, pixel data may be extracted from the medical image data item and stored as a pixel object in the storage component. In some cases, pixel data may be extracted from medical image data items prior to storage in LIM 104 . For example, the pixel data identified at 430 may be used to generate one or more pixel objects of the medical image data item.

The pixel objects may include one or more pixel objects for each sub-image in the medical image data item. For example, each sub-image may have a corresponding set of pixel objects stored in storage memory. When a request for a sub-image or a portion of a sub-image is received by LIM 104, LIM 104 may then retrieve the corresponding pixel object or objects. A pixel object(s) can then be provided to fulfill the request. Each pixel object may be stored at an individually addressable memory location in storage memory to facilitate retrieval of individual pixel objects.

In some cases, pixel objects may be stored independently in storage components. That is, the address in memory where each pixel object is stored can be determined independently of the address locations of other pixel objects. This may provide the flexibility to store pixel objects using distributed memory address locations.

In some cases, a collection of pixel objects for a sub-image may be stored in memory as a group. In this case, each collection of pixel objects can then be stored independently of the other collections of pixel objects.

In some cases, both the raw medical image data items and the extracted pixel data may be stored in the storage component. This ensures regulatory compliance while allowing direct retrieval of pixel data for individual sub-images and parts of sub-images without on-the-fly processing.

In some cases, the raw medical image data items may be stored in the long-term storage memory portion, while the extracted pixel data is stored in the cache portion. A cache can provide faster access to data stored within it. Accordingly, pixel data may be retrieved to satisfy medical image requests received by LIM 104 .

Alternatively, only raw medical image data items may be permanently stored. Alternatively, only the extracted pixel data may be stored in the storage component, and the original medical image data items may be discarded or stored in a remote image archive.

In some cases (ie, when the extracted pixel data/pixel objects are not permanently stored), the pixel data may be extracted from the medical image data item in response to a request from the viewer 108 . Additionally or alternatively, pixel data may be extracted from the medical image data item in response to a prefetch message from PACS 106 . Additionally or alternatively, LIM 104 may determine that a request for pixel data is expected (eg, because the pixel data/pixel object is related to a most recently requested pixel data/pixel object) and fetch the pixel data prior to receiving the request.

In this case, pixel data may be temporarily stored in LIM 104 (eg, in a cache). The pixel data can then be discarded/deleted after a period of time and/or once a storage threshold is reached.

In some cases, pixel data from the medical image data item may be analyzed to identify attributes of additional sub-images of the medical image data item. These additional sub-image attributes may then be stored as additional metadata of the medical image data item. For example, sub-image attributes may be used to provide an indication or estimate of the importance of a particular sub-image or sub-image region.

At 440, the large image manager can generate a representative data object of the medical image data item. Representative objects may be generated to represent pixel data and associated image metadata stored in LIM 104 . At 450 , at least some of the representative data objects may be stored in storage memory of PACS 106 .

Some of the representative objects may be generated in a DICOM compatible format. DICOM compliant representative objects can be transferred from LIM 104 to PACS 106 for incorporation into PACS 106 workflow.

The representative object generated at 440 may include at least one representative metadata object. Each representative metadata object may include metadata representing or reflecting characteristics of one of the sub-images in the item of medical image data.

Representative metadata objects may be generated by LIM 104 in a DICOM compatible format. The representative metadata objects can be transferred to PACS 106 and then used by PACS 106 in image management workflows. For example, a user accessing PACS 106 may receive data from a representative metadata object as part of a process for determining what medical images to request and/or requesting medical images.

The representative objects generated at 440 may also include overview objects. The overview object may be generated using both pixel data and metadata from the medical image data item. Each overview object may correspond to a medical image and/or a sub-image of a medical image. The overview object may include sub-samples of sub-images or pixel data for lower-resolution representations. Profile objects can also be generated and provided to PACS 106 in a DICOM compatible format.

In some cases, a pixel metadata object or fragment may also be generated. Each pixel metadata object may include metadata related to a particular pixel object generated from pixel data of the medical image data item. A pixel metadata object may include metadata identifying characteristics that define individual pixel objects. A pixel metadata object may also include relational metadata that defines relationships between pixel objects. For example, relational metadata can identify spatial relationships between multiple pixel objects.

In some cases, pixel metadata objects may be stored in LIM 104 . Because pixel objects are also stored in LIM 104, pixel metadata objects may not be needed unless or until a request for one or more pixel objects is received. Accordingly, pixel metadata objects can be stored in LIM 104 to reduce storage required by PACS 106 . As with pixel objects, in some cases pixel metadata objects may only be generated in response to a request (or in anticipation of a request or a prefetch instruction). Examples of image processing methods for general pixel objects, representative objects, and pixel metadata objects are described in further detail below with reference to FIG. 7 .

As described above, pixel objects and representative objects may be generated from metadata and pixel data identified in medical image data items. In some cases, metadata and/or pixel data are extracted from medical image data items and stored, eg, as pixel objects and representative objects.

In some cases, LIM 104 may store a map of medical image data items that indicates portions of the medical image data item that store metadata and pixel data, respectively. For example, LIM 104 may store a map of medical image data items in embodiments in which pixel objects are not permanently stored. This can facilitate subsequent generation of pixel objects and/or representative objects.

By storing medical image data items in LIM 104 external to the core workflow of PACS 106, unnecessary transmission or movement of medical image data items can be avoided since those data items are not directly related to the workflow. This prevents network saturation and increases PACS 106 availability for other potentially critical functions.

Representative objects provided to PACS 106 may be generated with a limited or restricted maximum data size such as, for example, 1MB or 10MB. The maximum data size can vary for different types of representative objects. For example, an overview object may have a maximum data size of 10MB, while a representative metadata object may have a maximum data size of 1MB. It should be noted, however, that the maximum data size may vary according to a particular system implementation.

Referring now to FIG. 5 , therein is shown a flowchart of a method 500 for managing medical images according to an example embodiment. In some cases, method 500 may be implemented as an image management application on an image management system, such as image manager 102 . In some cases, both methods 400 and 500 may be implemented as part of the same image management application.

Method 500 is an example of a process that may be used to provide pixel data in response to a request for medical images. Method 500 may generally be implemented as part of a system, such as system 100 , in which large medical image data items are stored external to the core components of PACS 106 .

At 510 , a request for medical images is received at PACS 106 . Medical images may be received from a computing device such as viewer 108 . A user, such as a clinician, interacting with viewer 108 may input a request to PACS 106 for a sub-image or a region of a sub-image.

Viewer 108 may transmit a request to PACS 106 for a sub-image or a portion of a sub-image. The request may include a requested image identifier corresponding to the unique identifier of the sub-image and/or sub-image region. Viewer 108 may determine the requested image identifier as part of a series of communications between viewer 108 and PACS 106 .

A user interacting with viewer 108 may initially select one or more representative objects stored on PACS 106 via a user interface provided by viewer 108 . For example, PACS 106 may transmit one or more metadata representative objects and/or overview objects to viewer 108 in message 118 . In some cases, a user may initially select or identify a particular patient at viewer 108 . PACS 106 may then provide the representative object corresponding to the patient to viewer 108 . Viewer 108 may then display metadata representative objects and/or overview objects in the GUI. A user may select one of the representative objects and initiate a request for the corresponding sub-image or sub-image region.

In some cases, PACS 106 may provide multiple overview objects corresponding to the same sub-image. For example, each of these overview objects may comprise pixel data of the same sub-image but at different resolutions. A user interacting with viewer 108 may initially view one or more overview objects to perform a preliminary viewing of medical images and/or sub-images. If the user determines that a higher resolution version is desired, the request may be initiated by the user, such as by selecting a button in the GUI of the viewer 108 to provide a high resolution sub-image and/or sub-image region. Request 120 may then be transmitted to PACS 106 .

At 520 , PACS 106 may determine that the requested medical image corresponds to a medical image data item for which pixel data is stored in LIM 104 . PACS 106 may determine that the requested medical image corresponds to a medical image data item stored in LIM 104 based on representative objects stored in PACS 106 . For example, viewer 108 may request medical images corresponding to a representative object selected by viewer 108 during the exchange of messages 118 . PACS 106 may determine from the metadata in the selected representative object that the medical image is a medical image data item stored in LIM 104 .

In some cases, PACS 106 may determine that the requested image corresponds to a large medical image data item based on the type of image requested. As described above, a medical image data item may be determined to be a large medical image data item based on the type of imager and/or the image processing used to generate the image. Accordingly, a request from viewer 108 may indicate a particular type of medical image. PACS 106 may then determine that the particular type corresponds to an image type defined as a large medical image data item.

In some cases, the request from viewer 108 may directly indicate that the requested medical image corresponds to a medical image data item stored in LIM 104 . For example, viewer 108 may determine that the requested medical image is stored in LIM 104 based on metadata in message 118 received from PACS 106 . Viewer 108 may then indicate in request 120 that the medical image corresponds to a medical image data item stored in LIM 104 (eg, by setting a large image flag or bit).

At 530, the location of pixel data corresponding to the requested medical image can be determined. In various embodiments, the location of the pixel data may be determined by LIM 104 , PACS 106 and/or viewer 108 .

In some cases, the address location corresponding to the pixel data of the requested medical image may be included in the metadata of one or more representative objects stored in PACS 106 . For example, the representative object(s) may include a URL indicating the location of the pixel data. When the representative object or the corresponding representative object is selected, the address location of the pixel data can be determined directly from the URL.

At 540, the request can be rerouted to the large image manager. The request may be rerouted to LIM 104 once PACS 106 has identified that the requested image corresponds to a medical image data item stored in LIM 104 . In some cases (e.g., where a small medical image data item is stored in PACS 106), the requested image may be provided directly by PACS 106 when PACS 106 determines that the requested image corresponds to a small image data item, e.g. Provided from database 232 .

The step of rerouting the request to the LIM 104 may occur before or after determining the memory location for the pixel data shown at 530 . For example, a memory location may be identified in a URL used by PACS 106 or viewer 108 to initiate the request. Alternatively, the rerouted request may simply identify the unique identifier of the requested image. In such cases, LIM 104 may determine the memory address location of the requested image based on the unique identifier identified from the rerouted request.

In some cases, requests may be rerouted to LIM 104 via PACS 106 . For example, viewer 108 may transmit a request for a medical image to PACS 106 , and PACS 106 may automatically reroute the request to LIM 104 .

In other cases, viewer 108 itself may reroute the request to LIM 104 . In some cases, viewer 108 may transmit a request for a medical image to PACS 106 , which may respond to viewer 108 with a rerouting message indicating that the request should be rerouted to LIM 104 . Viewer 108 may then reroute (ie, retransmit) the request to LIM 104 .

In some cases, viewer 108 may determine that the requested medical image is stored in LIM 104 based on metadata message 118 . In such cases, viewer 108 may automatically reroute its request to LIM 104 based on metadata from the representative object stored in PACS 106 .

At 550, LIM 104 can provide the pixel data of the requested image to the viewer. In some cases, LIM 104 may transmit pixel data directly to viewer 108 . Alternatively, LIM 104 may transmit the pixel data to PACS 106 , and PACS 106 may in turn provide the pixel data to viewer 108 .

In some cases, LIM 104 may simply retrieve the pixel data for transmission from pixel data already stored on LIM 104 . Alternatively, pixel data may be extracted from the corresponding medical image data item in response to the request, as described above. In some cases, LIM 104 may transmit only the pixel data of the requested image.

In some cases, LIM 104 may additionally transmit pixel metadata to viewer 108 . For example, LIM 104 may transmit the plurality of pixel objects and corresponding pixel metadata objects to viewer 108 where viewer 108 is capable of processing pixel metadata objects indicating relationships between the plurality of pixel objects. Alternatively, LIM 104 may only transmit pixel data corresponding to individual sub-images or regions of sub-images that are displayable in the GUI provided by viewer 108 .

Viewer 108 may be configured to communicate using the DICOM protocol used by PACS 106 . Accordingly, LIM 104 may generate a response that includes pixel data in a format compatible with the DICOM protocol used by viewer 108 .

A collection of large image data items

The following is a general description of systems and methods for collecting medical image data items and other features described herein, which may be used alone or in combination with one or more embodiments disclosed herein, including for managing medical image Systems and methods of data items, systems and methods for processing medical image data items, and systems and methods for communicating medical image data items. The following description contains various features of systems and methods for collecting medical image data items, which may be used alone or in any combination or sub-combination.

Collecting medical images from an image source location, such as an imager or a storage component associated with an imager, has a number of associated difficulties. Medical image data items are generated in a variety of different formats. Different imagers or imaging software products may each use different image formats to generate medical image data items. This can result in medical image data items with different data structures.

In some cases, the process used to generate the medical image data item at the image source location may not be known to external systems communicating with or accessing the image source location. Also, the file system (e.g. Windows, Linux, etc.) may vary depending on the image source location. As a result, the expected file structure of a medical image data item may not be immediately apparent.

Also, medical image data items are increasing in size. This may lead to complex data structures, as medical image data items may be stored in multiple medical image data files. The process of creating a medical image data file for a single medical image data item can also be a lengthy and time-consuming process. Therefore, determining when an item has finished generating can be difficult.

It may also be impractical or impossible to perform any processing at the image source location. For example, a central system configured to collect medical image data items may be limited to performing read operations on image source locations. This can make the process of collecting medical image data items even more unpredictable because the image collection system does not have any control over how and when medical image data items are generated, modified, stored, moved and/or deleted.

Image source locations may also have data management processes that complicate the process of collecting medical image data items. For example, the filenames given to medical image data files or items of medical image data at image source locations may repeat over time. After a period of time (eg, when a storage capacity threshold is reached), the image source location may routinely delete items from its image storage memory, and then reuse the same filename once the previously stored item has been deleted. Also, in some cases metadata (eg, timestamps) included in medical image data items may be unreliable or incorrect. Therefore, image collection systems may not be able to rely on filenames or timestamps to identify different medical image data items.

Embodiments of the image collection systems and methods described herein may allow medical image data items to be collected from a variety of different image source locations. Embodiments described herein may facilitate acquisition of medical image data items from isolated and distributed image source locations, such as medical imagers. This may allow centralized storage and/or management of a wider selection of medical image data items. Embodiments described herein may also provide a flexible image collection process configured to interact with and collect images from different types of image source locations.

Some embodiments described herein provide systems and methods for collecting medical image data items from one or more remote image storage devices or remote storage memories. In some cases, the systems and methods for collecting medical image data items may be used with the systems and methods described herein for managing medical image data items.

In some cases, remote storage memory or remote image storage memory may be provided by the medical imager. In some cases, the remote storage memory may also include other storage components, such as medical image archives. The remote image storage memory can store medical image data files generated in various data formats.

In some cases, the remote storage memory may not allow the LIM, PACS, or other external image collection system to create or modify any of the data or files stored thereon. Accordingly, the image collection system may be limited to monitoring/observing data generated and stored on remote storage memory and collecting/replicating that data.

The image collection system may define unique file identifiers for medical image data files identified on the remote storage memory. The unique file identifier may be separate from the file characteristic metadata assigned to the data file on the remote storage memory.

The image collection system may also define a collection corresponding to a medical image data item or a collection of expected files. Each collection of files may also be assigned a unique collection identifier. The unique collection identifier may include a unique file identifier assigned to each file or prospective file of the collection. The image collection system may use unique file identifiers and unique collection identifiers to ensure complete collection of medical image data items. This also allows the image collection system to independently identify the collected files, as the identifiers can remain independent of any deletion or modification of the files stored on the remote storage component.

Since individual medical image data files may be large in size, there may be a delay between the time a new file is identified and the time it has been completely generated or stored on remote memory. Thus, before copying the medical image data file from the remote storage memory, the image collection system may determine that storage of the medical image data file on the remote storage memory is complete. The image collection system may monitor file characteristics of the medical image data files on the remote storage memory to determine whether storage has completed.

The image collection system may also monitor the collection of individual files forming a collection of medical image data files defining medical image data items. Once all the medical image data files in the medical image data item have been copied by the image collection system, the image collection system may generate an indicator that the medical image data item is available. The indication may be used to initiate further processing, or the indication may retrieve or provide a medical image data item in response to a request.

The image collection system may also have various image collection criteria defined, such as image collection path, image priority criteria, security credential criteria, number of retries, image destination criteria. Image collection criteria may include both user-defined and system-defined collection criteria.

Referring now to FIG. 6 , there is shown a flowchart of a method 600 for collecting medical image data items according to an example embodiment. In general, method 600 may be used by a central image collection system to collect medical image data items from remote image storage memories associated with various image source locations. In some cases, method 600 may be implemented as an image collection application that forms part of an image management system, such as image manager 102 .

The central processor may be communicatively coupled to multiple remote image storage memory instances or components to collect medical image data items from different storage locations. The central processing unit can communicate with each remote image storage memory component to identify a medical image data file.

In some cases, the central processor may be limited to read operations on one or more of the remote image storage memory components with which the central processor is in communication. That is, the central processing unit may not be able to perform any operations or modifications to the data stored on the remote image storage memory component. The central processing unit may be limited to viewing and replicating data from remote image storage memory components.

At 610, a new medical image data file can be identified on a remote image storage memory. In some cases, the remote image storage memory may be a medical imager or a storage memory associated with the medical imager. In other cases, the remote image storage memory may be part of an image archive or external image management system.

The central processing unit may identify a collection of image data files stored on the remote image storage memory. The central processor may then identify new medical image data files from the set of image data files stored on the remote image storage memory. An example process for identifying a new medical image data file is shown in Table 1:

Table 1 - Example process for identifying new medical image data files

.

The central processing unit may assign a unique file identifier to each image data file. The central processor can associate the unique file identifier with a file characteristic or attribute of the image data file. For example, each unique file identifier may be associated with a particular combination of file characteristics, such as filename, timestamp, format, file size, and the like. In some cases, certain initial variable file properties associated with the unique file identifier may be updated until the central processor determines that storage of the image data file on the remote image storage memory component is complete (e.g., when the image data is being created file to account for changes in file size).

The unique file identifier may be defined by the central processor independently of the file characteristics of the image data file. For example, the unique file identifier may be independent of the filename and timestamp of the image data file on the remote image storage memory. This ensures that the CPU can uniquely identify files stored in the remote image store, for example in cases where the remote image store component re-uses filenames over time or may generate unreliable metadata (e.g. incorrect timestamps). Different medical image files on the component.

The central processor may then compare the unique file identifier of the image data file on the remote image storage memory component with the unique file identifier stored in the local/central storage memory associated with the central processor. The unique file identifiers stored on the local/central storage memory may correspond to medical image data files already stored in the local/central storage memory. A new file identifier can then be determined based on the comparison.

In some cases, the central processor may monitor the remote image storage memory location and identify when additional files are stored on the remote image storage memory. For example, a change to a collection of image data files (eg, files without a previously assigned unique identifier) may indicate that additional files have been stored. The additional file can then be identified as a new medical image data file.

The central processing unit may continuously (ie, continuously) monitor each remote image storage memory location. For example, the central processing unit may monitor each remote image storage memory location at periodic intervals to determine whether any new medical image data files have been stored. When a new medical image data file is identified, the central processor may assign the new medical image data file (ie its unique identifier) to the new file collection. The new set of files may include multiple medical image data files to be copied. The new file collection can be used to order the medical image data files to be copied to the central storage memory.

In general, the new file collection may be implemented in various configurations to allow the medical image data files in the new file collection to be arranged in a particular order. For example, the new collection of files can be implemented as a queue (eg, a messaging queue), such as the DetectedFileQueue shown in Table 1. Alternatively, the new collection of files may be sorted using database tables or other configurations known to those skilled in the art.

The new set of files can also be used to ensure that the medical image data files have completed generation at the remote storage storage before being copied to the local/central storage storage.

At 620, the central processor may determine that the new medical image data file is complete at the remote image storage memory. When the medical image data file has been generated on the remote image storage memory, the central processing unit can determine that the file is complete.

Since medical image data files may be large in size, generating the medical image data files may take place over a period of time. As a result, when the medical image data file is identified by the central processor at 620, the medical image data file may still be in the process of being generated. Accordingly, the central processing unit can monitor the file characteristics of new medical image data files to ensure that the files have completed storage/generation on the remote image storage memory before being copied. An example process for identifying completed medical image data files is shown in Table 2:

Table 2 - Example process for identifying completed medical image data files

.

As described above, the new medical image data files identified at 610 may be assigned to a new collection of files (eg, DetectedFileQueue). A new file collection (DetectedFileQueue) may be used to maintain a list of medical image data files stored on the remote image storage memory component that have not been copied to the central storage memory and have not been confirmed complete on the remote image storage memory. DetectedFileQueue can be arranged as a list stored in a central storage memory.

The files in the new file collection can be sorted based on various criteria. For example, files can be sorted based on a first-in-first-out (FIFO) order. This provides a simple arrangement for collecting medical image data files in order of detection by the central processor. Alternatively, other ordering criteria may be used, such as last-in-first-out (LIFO) or first-in-last-out (FILO) techniques.

In some cases, the new set of files may be sorted based on file attributes associated with each of the medical image data files. Each medical image data file may be assigned a priority value indicating the importance of copying the corresponding medical image data item to the central storage memory. Priority values can be assigned by various components such as imagers or CPUs. The priority value may be determined based on an expected request time period (eg, when a medical image data item is expected to be requested by a user of the image management system).

In some cases, the priority values assigned to medical image data files may change over time. For example, the priority value may increase over time to ensure that older medical image data files are copied before they are deleted from the source memory location. The priority value may also be updated based on input received from the central processing unit corresponding to the particular medical image data file. For example, an input indicating a follow-up appointment for a corresponding patient or a request for a medical image data file or related medical image data files may trigger an update of the priority value.

In some cases, other file attributes may also be used, such as a specific remote image storage memory location or the size of various medical image data files. In some cases, medical image data files may be initially grouped based on their assigned priority values. Groups of each file can then be arranged using the FIFO, LIFO or FILO criteria as previously described.

The central processing unit may determine that a new medical image data file has arrived at the "to be copied" position in the new file collection. For example, the "to be copied" location may be the top (ie, first) location in a new collection of files (eg, DetectedFileQueue), or the first multiple locations in a new collection of files.

When a new medical image data file reaches the "to be copied" position in the new file collection, the file characteristics of the medical image data file can be updated. The updated file properties may be compared to the file properties associated with the medical image data files from when they were allocated to the new file set. If the file characteristics have changed, the central processor may determine that storage of the new medical image at the remote image storage memory has not been completed (or at least may indicate that further review is required to ensure storage has been completed).

In some cases, the new file collection may also include a delay period between assigning the new medical image data file to the new file collection and when the new medical image data file is allowed to reach the "to be copied" location. This may allow the central processor time to monitor/identify any changes in the characteristics of the file after detection of the file.

An expected complete set of file properties may also be determined for a medical image data file. An expected complete set of file properties can be determined based on the data format of the medical image data file. The file properties of the new medical image data file may be compared to an expected complete set of file properties to ensure that storage of the medical image data file has been completed. When the file properties of the new medical image data file match the expected complete set of file properties, the new medical image data file may be determined as a completed medical image data file.

The configuration of the new file collection or DetectedFileQueue (and even the general method 600 ) can vary in different implementations of the file collection process. For example, a single new file set may be used for medical image data files from all remote image storage memory locations with which the central processor is in communication. This may provide a simplified sequence for the collection process, especially for implementations in which lower volume medical images are generated.

Alternatively, a separate new set of files may be used for each remote image storage memory location. For example, a separate new file set could be used for each medical imager in a healthcare facility. This can simplify determining which medical image data files to replicate in larger facilities where large numbers of medical image data files are generated.

In some cases, the specific configuration of new file collections may change over time. For example, the relationship between new file collections and remote image storage memory locations can be dynamically configured based on required performance and generated data volume size.

In some cases, each new set of files may be used for the same type of medical image data files. For example, there may be one or more new file sets for each imaging type and format of generated medical image data files. Various other configurations and combinations of new file collections can be used, as will be apparent to those skilled in the art.

Once it is determined that storage of the medical image data file has been completed on the remote image storage memory, the central processor may determine that the medical image data file is ready to be copied to the central storage component. The central processor may then copy the new medical image data file to the central storage memory at 630.

In some cases, new medical image data files may be assigned to a completed file collection (eg, CompletedFileQueue). Replication of new medical image data files may then occur based on the location of the medical image data files within the completed file collection. The completed document collection and sorting of the completed document collection can be performed in various ways, as explained above with reference to the new document collection. As with new file sets, completed file sets can also be configured for one or more remote imagers. In some cases, multiple completed file sets may be used by the central processor to arrange for copying of medical image data files from remote image storage memory locations.

An example process for copying a completed medical image data file is shown in Table 3:

Table 3 - Example process for copying completed medical image data files

.

The medical image data files may be copied to a central storage memory associated with the central processor. Medical image data files may be stored in the central storage memory in their original format. The unique file identifiers of the medical image data files may then be stored in the image source database on the central storage memory. The image source database may include a unique file identifier for each of the medical image data files copied from a particular remote storage memory location. In some cases, the image source database may be separated into different database records for different remote storage memory locations. In some cases, the image source database may provide combined records for all remote storage memory locations in communication with the central processor.

The image source database may also store related file information associated with each file identifier. The associated file information may include the source location of the medical image data file. The source location may include the date/time the medical image data file was collected and the full source address/location path from which the medical image data file was collected. The relevant file information may also include the current storage location of the medical image data file. The current storage location may identify the address at which the medical image data file is stored on the central storage memory.

Once the medical image data files are copied, the file identifiers can be moved from the completed file collection to the copied file collection (eg, CopiedFileQueue) on the central storage memory. The copied set of files may indicate that the medical image data files exist on the central storage memory to be accessed and/or used for further processing. The replicated file set may be implemented in various configurations as described above with reference to the new file set and the completed file set.

At 640, the central processor may determine an expected set of files corresponding to the copied image data files. The set of expected files may define a medical image data item of which the replicated image data file is a part. The expected set of files may include one or more medical image data files which together form a medical image data item generated on the remote storage memory.

A collection of files defining a medical image data item typically refers to a group of files generated from one imaging process/program (eg whole slide imaging scan, endoscopic video, CT scan, etc.). The collection of files defining medical image data items may vary according to the type of medical image. The set of files defining the medical image data item may also depend on the imager and/or imaging software used to capture the medical image and generate the medical image data item.

The central processor may determine the expected set of files based on characteristics of the copied image data files, such as file format, image type, imager type/brand, and imager software type, among others. For example, the central processing unit may determine the format of the copied image data file. The central processor may then determine the expected set of files based on the identified format of the replicated image data file.

For example, the Aperio digital pathology slide scanner can generate medical image data items as individual medical image data files with a .svs extension, which is in TIFF format and to which proprietary data and file attributes have been added. In this case, the collection of files of medical image data items may comprise a single medical image data file.

Other imagers may generate items of medical image data as named folders including subdirectories for files. The files subdirectory may include an index file and multiple pixel files, each pixel file storing a collection of complete pixel data for a particular class/grade of WSI images. Then, the medical image data files (ie index files and pixel files) in the subdirectories can constitute a collection of files of medical image data items.

In some cases, the central processor may determine the expected set of files based on previously replicated medical image data items. For example, the central processor may monitor a collection of files collected for previous medical image data items from the same remote image storage memory location. The central processor may then determine the expected set of files to correspond to the set of files copied from the same storage component (or generated by the same imager or type of imager) for the previous medical image data item.

In some cases, the central processing unit may determine the expected set of files prior to copying the image data files. Alternatively, the central processor may determine the expected set of files after the image data files have been copied to the central storage memory.

The central processor may define a unique set identifier for a set of files defining medical image data items. The unique collection identifier may identify a unique file identifier for each of the medical image data files in a particular medical image data item.

In some cases, the expected set of replicated image data files may have been predetermined based on previously replicated image data files of the same medical image data item. In such cases, the unique file identifiers of the copied image data files may be used to determine the expected set of files to identify the corresponding unique set identifiers of the expected set of files.

At 650, the central processor may determine that all of the files in the set of files identified at 640 have been copied to the central storage memory. For example, the central processor may compare the unique file identifiers defined in the unique set identifiers with file identifiers stored in the image source database. If all unique file identifiers match file identifiers in the image source database, the central processor can determine that all files have been copied.

Then, at 660, the central processor can generate an indicator that the medical image data item is available at the image management component. The indicator generated at 660 may indicate that the medical image data item is available for further processing by the central processing unit (eg, according to method 700 shown in FIG. 7 and described below).

The indicator may also indicate that the medical image data item may be accessed for review and/or transfer to an external system. In some cases, generating indicators may also introduce medical image data items (or corresponding representative objects) into workflows defined by the central processor or associated image management components.

In some cases, the central processor may determine that the collection of files for a particular item of medical image data was never created at the remote image storage memory component. This can happen, for example, if there was an error during the creation of the image data item and a rescan could be initiated. In such a case, the central processor may delete the medical image data file copied when the complete medical image data item will not become available.

After copying the medical image data item to the central storage memory, the central processor may ignore subsequent deletion of the data item on the remote storage memory. Thus, subsequent deletion of the data item on the remote image storage memory will not result in the data item being deleted from the central storage memory. This may ensure that the central storage memory controls storage and management of medical image data items independently of data management policies implemented by the imagers and/or remote image storage memories. For example, some remote image storage memory locations may periodically delete data items to account for limited storage capacity.

As described above, the unique file identifier may be generated by the central processor independently of the file characteristics of the medical image data file on the remote storage memory. Thus, when a new medical image data file having the same file name as a previously copied medical image data file is generated at the remote image storage memory, the central processor may assign a different unique file identifier to the new medical image data file document. This new medical image data file can then be copied to the central storage component as a single medical image data file.

In some cases, the remote storage memory may update/modify medical image data files stored thereon. The central processor can identify the updated medical image data file as a new data file and assign a new unique identifier to the updated medical image data file. The central processor may then copy the updated medical image data file to the central storage memory as a separate medical image data file from the previously copied data file.

For example, an imager associated with a remote image storage memory location may determine that a rescan or reimage is required for the same object (eg, the same tissue slide or the same patient). The results of such a rescan may be stored in the remote image storage memory with the same file name as the original scan or image data item. The central processor may identify the rescanned image data items as separate data items having different unique identifiers.

In some cases, remote image storage memory locations may store medical image data items and medical image data files in a format that is not compatible with the central image management system. For example, medical image data items may not be stored in a DICOM compliant format. Therefore, conventional PACS 106 may not be able to collect medical image data items from remote storage memory.

In such cases, the collected medical image data items may be stored in an external or additional image management component (such as LIM 104 ) in communication with PACS 106 . The image collection system may provide collected medical image data items for further processing by components of the image management system, such as LIM 104 . In some cases, the collected medical image data items may also be arranged in a processing set or processing file set to define an order in which the collected medical image data items are processed. Processing collections (or CopiedItemQueue) can be constructed and manipulated similarly to the collections described above, such as new files collection, completed files collection and copied files collection.

DICOM compliant representative objects may be generated from collected medical image data items and provided to PACS 106 for incorporation into PACS 106 workflow. For example, collected medical image data items may be processed using an embodiment of the method 700 for processing medical image data items.

Handling large image data items

The following is a general description of systems and methods for processing medical image data items and other features described herein, which may be used alone or in combination with one or more embodiments disclosed herein, including for managing medical image Systems and methods of data items, systems and methods for collecting medical image data items, and systems and methods for communicating medical image data items. The following description contains various features of systems and methods for processing medical image data items, which may be used alone or in any combination or sub-combination.

The systems and methods described herein for processing medical image data items may be implemented as part of an image management system, such as the image management system 100 . For example, embodiments of the method 700 for processing medical image data items may be implemented by the LIM 104 and the PACS 106 of the image manager 102 .

Medical image data items can be generated in different data sizes and in a variety of formats. In some cases (as with many large medical image data items), medical image data items may be generated in an imager-specific or proprietary format. As a result, medical image data items can be separated into different image management systems based on department or image type. These medical image data items may not be easily integrated into a centralized or combined image management system.

Many centralized healthcare systems (such as PACS, HIS, RIS, etc.) can use defined communication formats/protocols based on DICOM. However, since many medical image data items were not generated for compatibility with these DICOM-based systems, they remain isolated and may not be accessible through centralized healthcare management interfaces.

Even when generating DICOM-compliant images, it can be inefficient to incorporate entire large medical image data items directly into the image management workflow. The DICOM protocol is mainly concerned with standardized communication and data exchange between DICOM devices (often referred to as application entities). Performance and efficiency of communication and data transfer are not emphasized in workflows generated by DICOM based systems such as traditional PACS 106 .

While the standard DICOM communication protocol may allow medical image data items to be communicated between devices, frequent use of this method may be unreliable for large medical image data items with data sizes in the gigabyte and terabyte range or unsustainable. Accordingly, embodiments described herein can process medical image data items (both proprietary and DICOM formatted data items) to generate data that can facilitate storage, processing, and The retrieved export object. Some or all of these derived objects may be generated at a data size limited to a finite or limited maximum data size that is smaller (in some cases, by a factor of a thousand) than the data size of the medical image data item . For example, in some cases representative objects may be generated according to a limited maximum data size defined as 100MB. In other examples, representative objects may be generated according to a limited maximum data size defined as 10MB.

Embodiments described herein may also include systems and methods for processing medical image data items. Each medical image data item may include at least one set of medical images from the same imaging procedure (eg, from the same tissue sample, X-ray scan, endoscopic video, etc.). Each medical image collection defines sub-images from an imaging program. In some cases, a collection of medical images may define sub-images as a whole (ie, a single sub-image region). In other cases, a collection of medical images may have sub-images separated into multiple sub-image regions or portions.

Medical image data items may be processed to identify image metadata and image pixel data. In some cases, image pixel data may be extracted from medical image data items.

The pixel data may be used to generate a plurality of pixel objects which together comprise all the pixel data from the medical image data item. Each individual pixel object may include pixel data for a sub-image (eg, a z-plane or slice), a region within a sub-image (eg, a tile or sub-portion of a slice), or multiple regions within a sub-image. Pixel objects can be stored and retrieved independently to provide increased storage flexibility. Pixel objects may also be individually addressable in memory, providing the ability to access, modify or retrieve only desired sub-images or regions of sub-images.

Image metadata can be used to generate representative objects. Representative objects may include one or more collection representative objects. The representative objects may include set representative objects corresponding to each set of medical images and defining identifying characteristics of the sub-images defined by the set of medical images.

A collection representative object may be generated as a representative metadata object. Each representative metadata object may define identifying characteristics of the sub-images of the medical image defined by the medical image data item. In some cases, the representative metadata object may be generated in a format corresponding to the communication protocol used by the core component image management system, such as the DICOM protocol used by the core component of the PACS.

In some cases, representative objects may also include global representative metadata objects that define identifying properties of medical images as ensembles or groups.

An export object can also include one or more pixel metadata objects. Pixel metadata objects may include identification metadata that define identification characteristics of individual pixel objects. Pixel metadata objects may also include relational metadata that defines relationships between individual pixel objects. For example, relational metadata may define the spatial relationship between individual pixel objects. Relational metadata may indicate how sub-images or entire medical images may be reconstructed from pixel objects.

Representative objects may also include one or more overview objects. Each overview object may include both overview object metadata and overview pixel data. The overview pixel data may provide a representation of the medical image or sub-image at a lower resolution than defined by the corresponding pixel object. The overview object metadata may include a portion of image metadata corresponding to a representative medical image or sub-image. In some cases, the overview object may be generated in a format corresponding to the communication protocol used by the core component image management system, such as the DICOM protocol used by the core component of the PACS.

In some cases, the systems and methods for processing medical image data items may be used with the systems and methods for managing medical image data items and/or the systems and methods for collecting medical image data items described herein.

Referring now to FIG. 7 , there is shown a flowchart of a method 700 for processing medical image data items according to an example embodiment. In some cases, method 700 may be implemented as an image processing application within an image management system. For example, method 700 may be implemented using image manager 102 . Method 700 may also be implemented by an imager or medical imaging system.

At 710, a medical image data item can be received or identified. Medical image data items may be received from various external and/or internal image storage memory locations. In some cases, medical image data items may be identified in storage memory of a system implementing method 700 .

Medical image data items may be collected as part of an image collection process, such as the example process 600 described above with reference to FIG. 6 . Alternatively, medical image data items may be stored in an image archive, such as image archive 230 shown in FIG. 2 above. Medical image data items may also be stored in storage memory associated with the medical imager.

In some cases, medical image data items may already be stored within an image management system, such as by PACS 106/206 or by LIM 104/304. Alternatively, the medical image data items may be received from an external image management system, such as a PACS or an image manager associated with a different healthcare facility.

Items of medical image data may be received in various formats. In some cases, medical image data items may be received in non-DICOM formats. For example, medical image data items may be received in a proprietary format or a proprietary format based on open source formats such as TIFF, JPEG and BMP.

In some cases, medical image data items may be received in a DICOM compatible format. For example, medical image data items can be generated in DICOMWSI format.

A medical image data item typically includes one or more sets of medical images from the same medical imaging program. Typically, a collection of medical images defines sub-images of a medical image data item. Thus, each medical image set may comprise at least one sub-image object corresponding to a sub-image defined by the medical image set.

In some cases, each medical image set may correspond to a single sub-image object that defines the medical image set. Alternatively, a plurality of sub-image objects may correspond to a set of medical images, for example, wherein each sub-image object defines a region of a respective sub-image.

For example, a sub-image may refer to a frame of an endoscopic video medical image data item. A sub-image may also refer to a focal plane in a stack of medical images generated from a single tissue sample. A sub-image may also refer to a slice of a volumetric imaging series. In some cases, a medical image data item may include sub-images with different resolutions. Therefore, each set of medical images may have a corresponding resolution.

In some cases, a medical image data item may only comprise a single sub-image. Therefore, in some cases, a medical image data item may have only one medical image set. For example, a tissue sample slide digitized with a single focal plane may correspond to a medical image data item with a single sub-image (ie, focal plane).

The received medical image data items typically include one or more medical image data files. One or more medical image data files together form a medical image data item. One or more medical image data files are generated by the same medical imaging program.

A data item identifier or unique item identifier may be generated for the received medical image data item. For example, data item identifiers can be generated as unique collection identifiers.

At 720, image metadata and image pixel data can be identified in the medical image data item. For example, image metadata and image pixel data may be identified based on the file format of the received medical image data item (eg, as explained above with reference to parser 346 ).

The format of the medical image data item received at 710 can be identified. The number and type of medical image data files defining the medical image data item may be identified based on the determined format. Similarly, the medical image data file and the portion of the medical image data file storing the image metadata and image pixel data, respectively, may be determined based on the identified file format.

For example, medical image data items may be generated by an Aperio imager. Accordingly, the format of the medical image data item may be identified as a modified TIFF format corresponding to the Aperio imager. The medical image data item can then be parsed to extract attributes of the TIFF format/encoding. The extracted TIFF attributes can be further parsed to identify the definition of the Aperio format based on Aperio-specific attributes. Extracted TIFF attributes and Aperio-specific attributes are used to define image metadata and image pixel data.

Once the image metadata and image pixel data are identified, the image metadata and image pixel data can be extracted from the analyzed medical image data item. Derived objects such as pixel objects, representative objects, and/or pixel metadata objects may then be generated from the extracted image metadata and/or image pixel data. These derived objects can be used to facilitate storage, management and processing of medical images defined by medical image data items.

In some cases, the location of image metadata and image pixel data within a medical image data item may be stored for subsequent processing. For example, a map may be stored for a medical image data item that identifies locations of image metadata and image pixel data within the medical image data item.

In some cases, some or all of the exported objects may not be permanently stored. For example, a medical image data item may be permanently stored in its original format, and export objects may be generated as needed (eg, in response to a request for an image). Accordingly, storing a mapping of image metadata and image pixel data within or associated with a medical image data item may facilitate subsequent generation of derived objects.

In some cases, medical image data items may be processed to generate additional image metadata. The additional image metadata may then be included together with or in addition to the image metadata extracted from the medical image data item, the additional image metadata may be included.

In some cases, pixel data corresponding to one or more sub-images of an item of medical image data may be analyzed to identify sub-image metadata that may be included as part of additional image metadata. Pixel data can be analyzed to identify sub-image attributes. Sub-picture attributes may be used to provide an indication or estimate of the importance of a particular sub-picture. For example, an estimate of the importance of a sub-image may reflect an estimate of how likely the sub-image is to include pixel data of diagnostic value or importance.

The pixel data can be analyzed to identify potential tissue pixel data within the sub-images. For example, a process for identifying chest boundaries and chest windows within a medical image data item is described in US Patent No. 8,649,578, which is hereby incorporated by reference in its entirety. Sub-image metadata indicating the presence of the breast window and/or the position of the breast window in the sub-image may then be included in the extracted image metadata.

In some cases, the sub-image attributes may include a blemish attribute indicating that blemishes may be present in the sub-image or sub-image regions. In some cases, the sub-image attributes may include an artifact attribute indicating whether the sub-image includes image artifacts.

In some cases, the sub-image attributes may include focus or sharpness attributes indicating whether the sub-image is in focus or out of focus, as well as the level of sharpness of the sub-image. This may provide an indication as to whether a clinician is likely to observe diagnostic quality pixel data within the sub-image.

In some cases, the additional sub-image metadata may include user-specific metadata corresponding to the user requesting the medical image data item or the user who may be requesting the medical image data item. For example, pixel data from a sub-image may be compared to pixel data from a sub-image previously diagnostically accessed and used by a clinician. Subimage properties of subimages that are similar to those of subimages previously used for diagnosis may indicate an increased importance estimate.

At 730, a plurality of pixel objects can be generated from the image pixel data. Each pixel object corresponds to one of the sub-images in the medical image data item. Each sub-image may have at least one corresponding pixel object among the plurality of pixel objects.

Each pixel object typically includes at least a portion of image pixel data identified in the medical image data item. Each portion of image pixel data may be stored in one of the pixel objects generated at 730 . Accordingly, a plurality of pixel objects may provide a complete set of pixel data from the medical image data item received at 710 .

Each pixel object also includes a pixel object identifier having an identifying characteristic of the pixel object. A pixel object identifier may identify a data item identifier corresponding to the pixel object. The pixel object identifier may also identify the sub-image corresponding to the pixel object.

Each sub-image can have a corresponding pixel object. For example, where the medical image data item is a volume series, each slice may be stored as a single pixel object. Similarly, pixel objects can be generated for each slice of a CT/MR/US image, each frame of an endoscopic video, each slice of a cardiac scan, etc.

In some cases, multiple pixel objects may be generated for a sub-image. For example, pixel data corresponding to a sub-image may be divided into a plurality of sub-image regions. A pixel object may then be generated for each sub-image region, the pixel object including corresponding pixel data for that region. For example, pixel objects can be generated for each tile of a digitized pathology slide.

In some cases, an item of medical image data may include pixel data for one or more sub-images at multiple resolutions. One or more pixel objects can then be generated for each resolution of a particular sub-image.

At 740, a pixel object may be stored in a first memory location. In various embodiments, the first memory location may correspond to storage memory associated with various subsystems of the image management and healthcare document management system. Each pixel object may be stored at a corresponding pixel object address location in the first memory location. Each pixel object can be individually addressed at its corresponding pixel object address location.

For example, pixel objects may be stored in LIM 104 in some cases. Storing pixel objects in LIM 104 may remove pixel objects from the storage workflow of core PACS 106 components.

In some cases, pixel objects may be stored in storage memory associated with imager 110 . In some cases, pixel objects may be stored in an image archive, such as image archive 230 . In some cases, the first memory location may even be storage memory associated with PACS 206 .

In some cases, pixel objects may be stored as blocks or groups of pixel objects. The block of pixel objects may include all of the plurality of pixel objects stored adjacent to each other in the storage memory. This may facilitate storage and retrieval of consecutive pixel objects, for example by storing pixel objects adjacent to related pixel objects.

In some cases, pixel objects can be stored independently. That is, the address location in the first memory where each pixel object is stored may be determined independently of the address location of each other pixel object. This can provide flexibility when storing pixel objects to account for storage capacity considerations, such as without rearranging how data is currently stored.

In some cases, groups of pixel objects may be stored together but independently of other groups of pixel objects. For example, a group comprising pixel objects corresponding to one sub-image may be stored as a block in the first memory, but independently of groups comprising pixel objects corresponding to other sub-images.

In some cases, each pixel object may be individually addressable in the first storage memory. Individually addressable pixel objects may facilitate retrieval of pixel data in response to requests for sub-images or regions of sub-images. Pixel data corresponding to the requested sub-image or sub-image region may be directly recalled/retrieved based on the address location of the corresponding pixel object.

At 750, a plurality of representative objects can be generated for the medical image data item. A representative object may be generated from the image metadata and/or image pixel data identified at 720 . Typically, representative objects correspond to medical images and sub-images of medical image data items, but include less pixel data (eg, lower resolution images or sub-images) or no pixel data. A representative object may comprise representative metadata and/or representative pixel data that reflect the pixel data in the medical image data item, but with a smaller (usually substantially smaller) data size.

Each representative object also includes a representative object identifier that defines identifying characteristics of the representative object. A representative object identifier may identify a data item identifier corresponding to the representative object. The representative object identifier may also identify any sub-images or sub-image regions corresponding to the representative object. Thus, representative objects and pixel objects of the same medical image data item (and corresponding sub-image) may be determinable from the data item identifier and representative object identifier/pixel object identifier.

The representative objects can be used with data management and lifecycle workflows related to medical image data items, while the pixel objects remain stored in the first memory location. Representative objects may also be used to provide a user interface to users interested in medical image data items. For example, a representative object may identify available pixel data and pixel objects stored in the first memory location. Pixel objects can then be retrieved upon request to provide higher resolution medical images and sub-images. This can facilitate data management and workflow without the need to incorporate large data objects into the workflow.

The representative object can be stored in the second memory location. In some cases, the second memory location may be separate from the first memory location. For example, a first memory location may correspond to LIM 104 or image archive 230, while a second memory location corresponds to PACS 106/206.

Representative objects can be generated in a format compatible with the workflow of the image management system. For example, representative objects may be generated to be compatible with defined communication protocols, such as the DICOM protocol used by PACS 106/206. This facilitates the integration of representative objects into image management systems and workflows. In some cases, pixel objects may not be generated (or converted) to DICOM format before storage.

Representative objects can be generated with data sizes smaller than the medical image data items themselves to facilitate management within existing workflows. For example, in some examples, a representative object may be generated with a maximum data size of 100MB. In some examples, representative objects may be generated with a maximum data size of 10MB.

The representative object may include a representative metadata object generated from image metadata. Each representative metadata object may correspond to one of the sub-images of the medical image data item. Each representative metadata object may define identifying properties of the corresponding sub-image. Typically, the identifying properties of a sub-image are typically properties of that individual sub-image.

The identifying characteristics of the sub-images may include a data item identifier of the corresponding medical image data item, type of image, image resolution, available sub-regions (eg tiles), and a memory location identifying the first memory location. Identifying properties of sub-images may also include patient data, imaging study data, series data, imaging method data, and the like.

In some cases, the memory locations stored in the identification properties may include sub-image specific addresses indicating memory address locations of corresponding one or more pixel objects in the first memory location. In some cases, the memory location stored in the identification property may include sub-image region-specific address data for a pixel object corresponding to a particular region of the sub-image. For example, the representative metadata object may include a URL identifying a memory address location of the corresponding one or more pixel objects in the first memory location.

Representative metadata objects can be used with standard procedures of image management systems. For example, a plurality of representative metadata objects may include all metadata required to enable standard operations of the image management system, such as workflow, image lifecycle management, viewing, reporting, and the like.

In some cases, representative metadata objects may be generated in a DICOM-compatible format. For example, representative metadata objects can be generated as DICOM headers. This may allow representative metadata objects to be stored in a second memory location that is part of an image management system using the DICOM standard.

For example, representative metadata objects may be stored in PACS 106/206. Representative metadata objects can be used in workflows defined by the workflow activation component 114 . Multiple representative metadata objects can contain all DICOM metadata required for PACS 106/206 functions, such as workflow, image lifecycle management, viewing, reporting, etc. In some cases, a representative metadata object may be generated with a maximum data size of 1MB. In other words, each representative metadata object can be generated such that its data size is 1 MB or less.

In some cases, the representative object generated at 750 may also include at least one overview object of the medical image data item. Each profile object may include profile pixel objects and profile object metadata.

An overview pixel object may provide an overview of one or more sub-images of a medical image data item. For example, each overview pixel object may comprise a reduced resolution sub-image corresponding to one of the sub-images of the medical image data item. A reduced-resolution sub-image may have an overview resolution that is less than the resolution of the pixel object corresponding to the sub-image.

In some cases, each overview pixel object may have a data size that is one-thousandth that of the corresponding pixel object. In some cases, each overview pixel object may have a data size that is one-ten-thousandth that of the corresponding pixel object.

For example, a medical image data item may define a medical image having pixel data including 256000×256000 pixels. Corresponding overview pixel objects as small as 256x256 pixels can be generated. Thus, the reduced resolution sub-image is one millionth of the corresponding pixel data of the medical image data item.

In some cases, the at least one overview object may include a plurality of overview objects, wherein an overview object corresponds to each sub-image of the medical image data item.

In some cases, an overview object may include multiple overview objects corresponding to a particular sub-image, each overview object having a different overview resolution. That is, each of the plurality of overview objects corresponding to the same sub-image may have an overview pixel object with a different overview resolution. Each overview pixel object may provide a reduced resolution image or sub-image.

The overview object metadata may include a portion of image metadata corresponding to the portion of pixel data used to generate the overview pixel object (ie, corresponding to the sub-image represented by the overview pixel object).

In some cases, an overview object can be used as part of an image management system. For example, the overview object may provide a reduced resolution sub-image that may be displayed to a user interacting with the image management system, such as using the viewer 108 . The overview object may allow a user (physician, clinician, surgeon, etc.) to quickly perform a non-diagnostic review of the corresponding sub-image using the reduced resolution overview pixel object. In some cases, the overview object might even be sufficient for some aspects of the diagnostic review.

In some cases, an overview object may be generated with a maximum data size of 10MB. In other words, each overview object can be generated such that its data size is 10MB or less.

The overview object metadata may also include a list of representative metadata objects corresponding to the same medical image data item. The overview object metadata may list each of the representative metadata objects corresponding to the same medical image data item. The overview object metadata may provide a link or URL to the listed representative metadata objects. The overview object metadata may also identify all other overview objects available for the same medical image data item.

Similarly, the representative metadata objects may each comprise a list of all overview objects for the same medical image data item. The representative metadata object may also identify all other representative metadata objects available for the same medical image data item. Thus, a user interacting with the image management system may select one of the representative metadata objects and/or overview objects and be provided with identification information of all related overview objects and/or representative metadata objects.

In some cases, accessibility criteria can be defined for profile objects. For example, accessibility standards may restrict or restrict access to certain profile objects based on user roles defined within the image management system. Accessibility criteria can control access to specific profile objects based on a user's role. For example, accessibility standards may provide some users with access to only overview objects with lower resolution overview pixel objects, while other users may have access to higher resolution overview objects (or all overview objects).

The profile object's profile object metadata can be used in conjunction with accessibility criteria and the user's defined role to determine whether the user has access to the profile object. In some cases, the existence of a particular overview object may not be visible to users unless their role is eligible to access the overview object based on the particular overview object and the overview object metadata of the overview accessibility criteria. In some cases, the accessibility criteria may provide a user with access to only one resolution of an overview object of medical image data based on the user's role.

Accessibility standards may define the maximum accessible data size of an object accessible to a particular user. Users requesting overview objects or pixel objects may then be limited to viewing overview objects or pixel objects whose data size is smaller than the maximum accessible data size. For example, for a doctor/clinician user, the maximum accessible data size may be defined as 10MB, while for a reading pathologist, the maximum accessible data size is unlimited.

In some cases, overview objects can be generated in a DICOM compatible format. This can facilitate the integration of profile objects into image management systems (such as PACS 106/206) operating using the DICOM protocol. For example, overview objects can be generated as DICOM secondary capture objects.

In some cases, representative metadata objects may be omitted. For example, an overview object can be generated to include overview object metadata that also includes all metadata required for integration into image management systems and workflows. The overview object metadata may then be defined to include identifying characteristics of corresponding sub-images of the medical image data item.

Alternatively, the overview object can be omitted. In such cases, the representative metadata objects can be used both in the image management workflow and in providing information for the user interface. In such cases, however, the user may not be able to perform an initial viewing of the sub-image without requesting corresponding pixel data stored in the first memory location.

At 760, at least some of the representative objects may be stored in a second memory location. The representative metadata object and/or the overview object may be stored in a second memory location separate from the first memory location in which the pixel object is stored.

The second memory location in which the representative object is stored may be a storage memory associated with an image management component operating with a defined communication protocol. For example, representative objects may be stored in PACS 106 using the DICOM protocol.

In some cases, the second memory location may correspond to a fast-access storage component (e.g., a cache), while the first memory location is a slower storage component (e.g., an archival or long-term storage component) that may be located away from the processing of the image management system. ). For example, the second memory location may be stored locally at the image management system while the first memory location is external to or remote from the image management system. This can allow representative objects to be easily and quickly retrieved for ongoing image management processes and workflows, where pixel objects can be retrieved as needed.

In some cases, pixel metadata objects may also be generated for medical image data items. A pixel metadata object may correspond to a collection of medical images and a corresponding pixel object.

A set of at least one pixel metadata object may be generated, where each set corresponds to one of the sets of medical images. Each set of pixel metadata objects may include metadata identifying identifying characteristics defining the pixel objects corresponding to the set of medical images. Each set of pixel metadata objects may also include relational metadata defining spatial relationships between pixel objects corresponding to that set of medical images. Relational metadata may also include overall relational metadata that identifies spatial relationships between pixel objects of different (eg, adjacent) medical image sets/sub-images.

For example, medical image data items may be generated as DICOM WSI data items. A DICOM WSI data item may include binary data representing up to 11 million sub-images or sub-image regions (eg, tiles). Binary data can be included in the text array adjacent to the pixel data. Binary data can be extracted and used to define pixel metadata objects. A pixel metadata object can be generated as a DICOM fragment (part of a DICOM header object).

A pixel metadata object may be stored in the first memory location (or another memory location). Since the pixel metadata objects include image metadata defining the spatial relationship between the pixel objects stored in the first memory location, they may not be required for the functionality of the image management system unless the corresponding pixel objects are requested. Thus, pixel metadata objects can be stored outside of the core components of the image management system.

In some embodiments, pixel objects, representative objects, and/or pixel metadata objects may be used with an image management system, such as system 100 shown in FIG. 1 . In such an embodiment, the first memory location may correspond to LIM 104 and the second memory location may correspond to PACS 106 .

The DICOM conversion process 700 can then be performed primarily by the LIM 104 or an external image processing system. The pixel objects generated at 730 may be stored in LIM 104 . LIM 104 may transmit representative metadata objects and/or overview objects to PACS 106 as part of metadata message 116 or in a subsequent message or series of messages. The PACS 106 can then incorporate the representative metadata objects and/or profile objects into the workflow defined by the workflow activation component 114 .

In some examples, the set of representative objects generated by LIM 104 may include both representative metadata objects and overview objects. The collection of representative objects may be provided to a core image management system, such as PACS 106 . The profile object can be generated as a regular DICOM object including both profile pixel data and profile object metadata. A representative metadata object may include only image metadata and possibly a URL to corresponding pixel data in LIM 104 .

A user of viewer 108 may interact with PACS 106 to view and request data related to medical image data items. Viewer 108 may include representative metadata objects and/or overview objects in a graphical user interface presented to the user based on the objects received from PACS 106 in viewer message 118 . A user may initiate a request 120 for a medical image, for example by selecting one of the representative objects displayed in the user interface. In some cases, the user may review one or more of the overview objects prior to or instead of requesting pixel data for the medical image.

If request 120 corresponds to a medical image data item having pixel data stored in LIM 104 (eg, a large medical image data item), request 120 may be rerouted to LIM 104 . LIM 104 may then transmit an image response message 124 to viewer 108 with a pixel object including the requested pixel data.

In some cases, viewer 108 may be equipped to display multiple pixel objects. In such cases, LIM 104 may also include the required pixel metadata objects in response 124 .

In other cases, a user of viewer 108 may request related pixel objects (eg, neighboring pixel objects) after receiving the pixel objects in response 124 . LIM 104 may then use the stored pixel metadata objects (ie, relational metadata) to identify the correct pixel objects to provide to viewer 108 .

Embodiments described herein may provide image processing systems and methods that convert medical image data items into a plurality of derived objects. Export objects can include pixel objects and representative objects. In some cases, an export object may also include a pixel metadata object. Representational objects can be integrated into general image management and lifecycle workflows, while pixel objects are stored outside of the core image management system. This can provide high efficiency and throughput for managing objects, while still allowing high-resolution pixel objects to be accessed on request.

In general, derived objects such as pixel objects, representative metadata objects, representative overview objects, pixel metadata objects and raw medical image data items may each include an item identification corresponding to a medical image data item. This allows each of the objects and files related to the same medical imaging procedure to be easily identified within the imaging system.

In some cases, raw medical image data items may be stored in their original state to prevent modification. For example, a digital signature can be defined for each raw medical image data item. A digital signature may indicate that the original medical image data item has not been modified from its original state. The digital signature can be configured to be modified or deleted if any modification is made to the original medical image data item. In such cases, the derived object may be generated by reading and/or extracting the image metadata and image pixel data without modifying the original image metadata or image pixel data.

passing large image data items

The following is a general description of systems and methods for communicating medical image data items and other features described herein, which may be used alone or in combination with one or more embodiments disclosed herein, including for managing medical image Systems and methods for data items, systems and methods for collecting medical image data items, and systems and methods for processing medical image data items. The following description contains various features of systems and methods for communicating medical image data items, which may be used alone or in any combination or sub-combination.

Referring now to FIG. 8 , therein is shown a flowchart of a method 800 for managing medical images according to an example embodiment. In general, method 800 may be used to transfer medical images between remotely located image storage locations.

For example, method 800 may be implemented using an image management system, such as system 100 shown in FIG. 1 , PACS 204 shown in FIG. 2 , and/or LIM 304 shown in FIG. 3 . In some cases, method 800 may be implemented as an image delivery application on an image management system, such as image manager 102 .

Alternatively, method 800 can be implemented between two PACS 206 systems or between any two image storage components.

At 810, a request for a large medical image data item can be received from an external system. For example, the request may be received by PACS 106/206 or LIM 104/304. The request may identify a defined communication format/protocol (eg, DICOM) for the requested medical image data item.

At 820, pixel objects of the large medical image data item can be retrieved from the first storage location. For example, pixel objects may be stored in LIM 104 or image archive 230 external to the core component of the image management system receiving the request. Typically, a pixel object defines pixel data for a sub-image or sub-image region within a medical image data item. A collection of pixel objects of a large medical image data item typically includes all pixel data of the medical image data item.

At 830, a representative object of the large medical image data item can be retrieved from the second storage location. For example, the representative object may be stored in a storage component forming part of the core image management system receiving the request, such as PACS 106/206. Typically, the representative object includes metadata associated with the medical image data item as a whole or metadata associated with the sub-image as a whole. For example, metadata included in a representative object may reflect resolution level, patient data, image type, imager type, available sub-images, and the like.

Optionally, at 840, a pixel metadata object for the large medical image data item may be retrieved from the first storage location. The pixel metadata object may be stored along with the pixel object, eg, in the LIM 104 or image archive 230 external to the core image management system receiving the request. A pixel metadata object can define the characteristics of individual pixel objects and the spatial relationship between individual pixel objects. In some cases, eg, where the metadata is stored as part of the representative object retrieved at 830, a pixel metadata object may not be needed.

At 850, a large medical image data item can be reconstructed using pixel data and metadata from the pixel object, the representative object, and optionally the pixel metadata object. A large medical image data item may be reconstructed as a combined medical image data object comprising metadata from a representative object (and optionally a pixel metadata object) and pixel data from a pixel object. A combined medical image data object can be defined according to the requested transmission format (eg DICOM).

At 860, the reconstructed combined medical image data object can be transmitted to an external system, an example of which is shown in FIG. 9 .

Referring now to FIG. 9 , there is shown a block diagram 900 illustrating the transfer of medical image data items between a first image system 960 and an external system 970 according to an example embodiment. For example, the medical image data item transmitted at block 900 may correspond to the combined medical image data object generated at 850 .

In some cases, the first imaging system 960 may be the PACS 106/206 associated with a first healthcare provider or facility, while the second imaging system 970 is the PACS 106 associated with a second external healthcare provider or facility /206. Alternatively, the first imaging system 960 may be the LIM 104 and the second imaging system is the PACS 106/206. Various other systems may be used as the first image system 960 and the external system 970 .

Block diagram 900 illustrates an example image transfer process that may be used to implement step 860 of method 800 . Block diagram 900 is an example of how DICOM cascading may be used to transfer large items of medical image data between image management systems. Concatenation can be used to separate a single DICOM object into multiple DICOM objects or concatenations.

At 962 , a large image data item is generated at the first image management system 960 . As noted above, large image data items may be generated in steps 810-880 of method 800 as described above. Alternatively, the large image data item may already be stored on the first system 960 (eg, in the storage memory of the LIM 104). In some cases, large image data items can be generated according to the DICOM standard.

At 964, the large image data item is broken down into multiple DICOM concatenations. A collection of DICOM concatenations is then provided for transmission at 966 .

At 968 , an association is established between the first image system 960 and the external system 970 . Establishing the association may involve establishing a network communication channel between the first imaging system 960 and the external system 970 . A low-level protocol for a DICOM network connection may be defined using an exchange 972 of association establishment messages (i.e., a DICOM handshake) to ensure that the first image system 960 and the second image system 970 are compatible and transmit data in a well-defined format and order .

Once the association is established, the set of concatenations 966 is transmitted to the external system 970 via a transport message(s) 974 . External system 970 completes receiving the concatenated set at 976 . The external system 970 may then reconstruct the large image data item from the concatenated collection at 978 and store the reconstructed large image data item in the local storage memory at 980 . At 982 , the external system 970 may transmit a confirmation message 984 to the first system 960 indicating successful delivery of the large image data item.

The present invention has been described herein by way of example only, and numerous specific details have been set forth herein in order to provide a thorough understanding of the exemplary embodiments described herein. However, it will be understood by those of ordinary skill in the art that, in some instances, these embodiments may be practiced without these specific details. In other instances, well-known methods, procedures, and components have not been described in detail so as not to obscure the description of the embodiments. Various modifications and changes may be made to these exemplary embodiments without departing from the spirit and scope of the invention, which is defined only by the appended claims.

Claims (11)

1. A system for managing medical image data items, said system comprising: a picture archiving and communication system (PACS) comprising an image storage memory configured to store medical image data items, and the PACS is communicatively coupled to at least one viewing station; a large image manager component in communication with the PACS and with at least one medical imager, the large image manager component including a large image storage memory; in, The large image manager component is configured to: identifying large medical image data items generated by at least one medical imager; storing pixel data from each of the large medical image data items in the large image storage memory; generating at least one representative data object for each of the large medical image data items; and transmitting at least one representative data object of each of the large medical image data items to the PACS; and The PACS is configured to: At least one representative data object for each of the large medical image data items is stored in an image storage memory. 2. The system of claim 1, wherein: The PACS is configured to: receiving a request for a particular medical image from one of the observation stations; determining that the particular medical image corresponds to one of the large medical image data items stored in the large image storage memory; and Reroutes requests for specific medical images to the Large Image Manager component; and The large image manager component is configured to provide, in response to the rerouted request, specific pixel data corresponding to a specific medical image from the pixel data stored for the corresponding large medical image data item to the viewing station. 3. The system of claim 1, wherein: The large image manager component is further configured to: determining that a new large medical image data item has been generated at one of the medical imagers, the new large medical image data item comprising pixel data defining the one or more sub-images and metadata corresponding to the one or more sub-images; extract metadata from new large medical image data items; generating at least one representative data object corresponding to the new large medical image data item to include at least a portion of the extracted metadata; and transfer at least one representative data object to the PACS; and The PACS is configured to store at least one representative data object in an image storage memory. 4. The system of claim 1, wherein: The large image manager component is configured to: determining that a new large medical image data item has been generated at one of the medical imagers, the new large medical image data item comprising pixel data defining the one or more sub-images and metadata corresponding to the one or more sub-images; extract pixel data from new large medical image data items; and Store the extracted pixel data in a large image storage memory. 5. The system of claim 1, wherein: a large medical image data item comprising a plurality of sub-images of the series of medical images; and The large image manager component is configured to: storing image-specific pixel data for each of a plurality of sub-images in a series of medical images, wherein the image-specific pixel data for at least some of the sub-images is stored separately from image-specific pixel data for other portions of the sub-images . 6. The system of claim 1, wherein: the PACS is configured to communicate using a PACS communication protocol; the medical imager is configured to generate the large medical image data item in a format different from that defined by the PACS communication protocol; and The large image manager component is configured to generate at least one representative data object in a format defined by the PACS communication protocol. 7. A method of managing medical image data items using a picture archiving and communication system (PACS) comprising an image storage memory configured to store medical image data items and a PACS comprising a large image storage memory A large image manager component of communication, the method comprising: identifying large medical image data items generated by at least one medical imager; storing pixel data corresponding to each of the large medical image data items in a large image storage memory; generating, by the large image manager component, at least one representative data object for each of the large medical image data items; transmitting, by the large image manager component, at least one representative data object for each of the large medical image data items to the PACS; and At least one representative data object for each of the large medical image data items is stored in the PACS. 8. The method of claim 7, further comprising: A request for a specific medical image is received by the PACS from the viewing station; determining that the particular medical image corresponds to one of the large medical image data items stored in the large image storage memory; Reroutes requests for specific medical images to the Large Image Manager component; and In response to the re-routed request, the particular pixel data corresponding to the particular medical image is provided by the large image manager component to the viewing station from the pixel data stored for the corresponding large medical image data item. 9. The method of claim 7, further comprising: determining that a new large medical image data item has been generated at one of the medical imagers, the new large medical image data item comprising pixel data defining the one or more sub-images and metadata corresponding to the one or more sub-images; extract metadata from new large medical image data items; generating at least one representative data object corresponding to the new large medical image data item to include at least a portion of the extracted metadata; and At least one representative data object is stored in the PACS. 10. The method of claim 7, further comprising: determining that a new large medical image data item has been generated at one of the medical imagers, the new large medical image data item comprising pixel data defining the one or more sub-images and metadata corresponding to the one or more sub-images; extract pixel data from new large medical image data items; and Store the extracted pixel data in a large image storage memory. 11. A computer program product comprising a non-transitory computer readable storage medium storing computer executable instructions for configuring a processor to perform a method of managing medical image data items, wherein the method is defined according to claim 7.