CN108876889B - In-situ volume rendering method - Google Patents

Abstract

The invention discloses an in-situ volume rendering method, which is used for in-situ preprocessing of data volume rendering during large-scale scientific computing, so as to greatly reduce the amount of data transmission and storage; body drawing. The present invention maps the original data into a volume depth image at a computing node, and the image is a direct volume rendering result that retains the depth image. Since it is difficult for users to intervene interactively during in-situ calculation, the main parameters of the drawing are automatically optimized by particle swarm optimization. The user can set the desired compression rate and rendering time, and the algorithm finds the rendering parameters to achieve the best rendering effect accordingly. This method can reduce the amount of transmitted data by 1 to 3 orders of magnitude for large-scale scientific computing. Tested on several large-scale scientific computing applications to verify the effectiveness of the method.

Description

An in-situ volume rendering method

technical field

The invention relates to the field of visualization, in particular to an in-situ volume rendering method.

Background technique

With the rapid development of computer hardware technology and numerical simulation methods, large-scale scientific computing has begun to attempt exascale computing (10 ¹⁸ , 10 billion billion times), but the current large-scale computer's disk read and write speed is at least 4 slower than CPU computing speed By 5 orders of magnitude, data reading and writing has become the main speed bottleneck for scientific computing and scientific data analysis.

In order to solve this problem, the current main method is to sparse and sample the output data in time and space to reduce the scale of the output data, but this method will inevitably lose transient and small-scale features, which is a great waste of the original calculation results. . In order to make full use of all computational data without increasing data throughput, the concept of in situ analysis and visualization has been proposed in recent years in academia and industry. The core idea is that when the data is obtained from the calculation and simulation, it is immediately analyzed in situ (without transmission and storage), and then the original data is discarded, and only the results of these analysis and visualization are output.

Although the idea of in situ analysis and visualization is very straightforward, the actual implementation is very difficult, and the real application is still very rare. The main problem is that because data analysis and visualization is often an exploratory process, it is often impossible to determine appropriate methods and parameters "in situ" of the computation.

SUMMARY OF THE INVENTION

In view of the deficiencies of the prior art, the present invention proposes an in-situ volume rendering method, which uses volume depth images as intermediate data, and uses an optimization algorithm for automatic parameter setting, finally realizing interactive volume rendering of super large-scale scientific data. The specific technical solutions are as follows:

An in-situ volume rendering method, characterized in that the method comprises the following steps:

(1) Volume depth image generation: In the in-situ computing node of large-scale scientific computing, the data of the calculation result is processed, and the processing method is as follows:

(1.1) Send a ray from the viewpoint O to each pixel of the drawing area (W, H), and calculate the intersection of each ray and the volume data bounding box, where W and H represent the width and height of the drawing area, respectively;

使用三线性插值方法，从周边数据插值得到采样值；(1.2) Sampling at equal intervals from the incident point to the exit point of the bounding box along the ray, and the sampling interval is denoted as

Use trilinear interpolation method to interpolate sample values from surrounding data;

(1.3) Use the transfer function F to obtain the color value c and the opacity value α of the sampling point;

(1.4) When the color similarity of adjacent sampling points on the same ray is less than a specific threshold δ, they are merged. The merged sampling point set is called a super segment, and its attributes include start position, end position, color value , transparency, the combination of all super fragments is called the volume depth image, which contains the depth information and color information of the data volume rendering when observed from a specific viewpoint;

(2) Volume depth image drawing: transfer the volume depth image obtained in step (1.4) to the drawing node for drawing, and the drawing method of the volume depth image is:

(2.1) Expand each super fragment into a frustum by using the pixel position and the position of the viewpoint during generation;

(2.2) Sort all frustums by depth according to the current drawing viewpoint;

(2.3) After sorting, all frustums are drawn directly, and the transparency of each frustum is corrected when drawing: the transparency is based on the length of the frustum and the length of the line segment through which the current line of sight passes through the frustum. The relationship is determined using the following formula:

where η is the transparency value of the frustum, s is the length of the frustum, s' is the length of the line segment where the current line of sight passes through the frustum, and η' is the corrected transparency value;

(3) Automatic optimization of operating parameters of volume depth image

对其进行寻优；The key operating parameter combination for volume depth image generation is:

optimize it;

(3.1) Select a set of parameter groups in the parameter space to generate volume depth images, and obtain the corresponding volume depth image generation time t and volume depth image compression rate c;

(3.2) Use the volume depth image to draw the volume depth image to obtain the final image, and calculate the quality q of the final image;

(3.3) Bring the t, c, q values into the evaluation function to obtain the evaluation value of this group of parameters, and update the global optimal parameter group according to the evaluation value. The evaluation function formula is as follows:

是数据压缩率的评估函数，两者公式如下：In the formula, k ₁ , k ₂ , and k ₃ are the generation time of the volume depth image, the data compression rate of the volume depth image, and the weight of the volume depth image rendering quality, and satisfy k ₁ +k ₂ +k ₃ =1, ψ(t ) is the evaluation function of the generation time,

is the evaluation function of the data compression rate, and the two formulas are as follows:

In the formula, α and β are the two proportional division points of the generation time and the compression rate, respectively, and their values are in the (0, 1) interval, which are used for regulation.

(3.4) End the optimization according to the termination condition, and obtain the optimal parameter group; otherwise, go back to (3.1).

(4) In the subsequent simulation calculation, the volume depth image is generated according to the optimal parameter group.

Preferably, the volume depth image generation time t in (3.1) is obtained by the following formula:

where t _i is the time taken by the i-th computing node to generate the volume depth image, and N is the number of computing nodes.

Preferably, the volume depth image compression rate c in (3.1) is obtained by the following formula:

In the formula, D _i is the data size of the volume depth image generated by the i-th computing node, and D _raw represents the size of the original data.

Preferably, the quality q of the image in the step (3.2) is obtained by the following formula:

n is the number of image pixels, X _obs,i is the color value of the ith pixel of the image drawn using the volume depth image, and X _model,i is the color value of the ith pixel of a reference image. Since each color value is a three-dimensional vector, (X _obs,i -X _model,i ) is calculated using the Euler distance formula. The smaller the value, the closer the two images are.

Preferably, the step (3) uses particle swarm optimization for optimization, and the process includes:

(a) Randomly generate J points, each point includes a set of position information and velocity information, the position information is a set of volume depth image parameter groups, the parameter group is randomly generated in the parameter space, and the velocity value is (-1,1) is randomly generated in the parameter space of ;

(b) According to the parameter group of each point, generate a volume depth image, obtain the generation time t and the compression rate c, use the generated volume depth image data to draw an image, and compare with the reference image to calculate the image quality q;

(c) According to t, c, q, bring into the evaluation function E to calculate the evaluation value e, and update the historical optimal solution and the global optimal solution of the point according to e;

(d) Update the velocity and position information of each point according to the historical optimal solution and the global optimal solution.

(e) Determine whether the iteration ends or not by judging whether the Euler distance between more than half of the points in the point set and the global optimal solution is less than a certain threshold ε. If it does not meet the standard, go back to step (b), otherwise end the optimization, and the global optimal solution is the optimization result.

Compared with the prior art, the beneficial effects of the present invention are as follows:

1. The use of volume depth images can greatly reduce the original data;

2. The use of volume depth image rendering can achieve high-quality image results and certain visual interaction effects;

3. The automatic optimization of volume depth image parameters based on particle swarm algorithm can quickly and well search for the optimal parameter group in a large-scale and continuous parameter space.

Description of drawings

Figure 1 In-situ volume rendering working mode;

Figure 2 is a schematic diagram of volume depth image generation;

Figure 3 is a schematic diagram of volume depth image rendering;

Figure 4 is a flowchart of automatic optimization of parameters based on particle swarm optimization;

Figure 5 DNS turbulent in situ volume rendering results.

Detailed ways

The present invention will be described in detail below according to the accompanying drawings and preferred embodiments, and the purpose and effects of the present invention will become clearer. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

As shown in Figure 1, the in-situ volume rendering algorithm is divided into two parts, one part runs on the computing simulation node in the supercomputer to convert the original calculation data into the intermediate representation of the depth image, and the other part runs on the rendering server equipped with the graphics card Above, the volume depth image is drawn according to the viewpoint to realize the volume rendering effect. The parameter selection of the volume depth image is realized by the parameter optimization algorithm, which only runs once in the first time step of the calculation. After the parameters are selected, the subsequent runs do not change. The specific implementation is as follows:

An in-situ volume rendering method, the method comprises the following steps:

(1) Volume depth image generation (as shown in Figure 2): In the in-situ computing node of large-scale scientific computing, the data of the calculation result is processed, and the processing method is as follows:

(1.1) Send a ray from the viewpoint O to each pixel of the drawing area (W, H), and calculate the intersection of each ray and the volume data bounding box, where W and H represent the width and height of the drawing area, respectively;

使用三线性插值方法，从周边数据插值得到采样值；(1.2) Sampling at equal intervals from the incident point to the exit point of the bounding box along the ray, and the sampling interval is denoted as

Use trilinear interpolation method to interpolate sample values from surrounding data;

(1.3) Use the transfer function F to obtain the color value c and the opacity value α of the sampling point;

(1.4) When the color similarity of adjacent sampling points on the same ray is less than a specific threshold δ, they are merged. The merged sampling point set is called a super segment, and its attributes include start position, end position, color Value, transparency, the combination of all super fragments is called the volume depth image, which contains the depth information and color information of the data volume rendering when viewed from a specific viewpoint;

(2) Volume depth image rendering (as shown in Figure 3): The volume depth image obtained in step (1.4) is transmitted to the rendering node for rendering. The volume depth image is generally 1 to 3 orders of magnitude smaller than the original data, and can be transmitted to the Drawing node drawing can also be permanently stored in the disk array for post-processing interactive analysis. The drawing method of the volume depth image is:

(2.1) Expand each super fragment into a frustum by using the pixel position and the position of the viewpoint during generation;

(2.2) Sort all frustums by depth according to the current drawing viewpoint;

(2.3) After sorting, all frustums are drawn directly, and the transparency of each frustum is corrected when drawing: the transparency is based on the length of the frustum and the length of the line segment through which the current line of sight passes through the frustum. The relationship is determined using the following formula:

where η is the transparency value of the frustum, s is the length of the frustum, s' is the length of the line segment where the current line of sight passes through the frustum, and η' is the corrected transparency value;

(3) Automatic optimization of operating parameters of volume depth image

The volume depth image is carried out in situ, and its optimal operating parameters are difficult to manually set. An automatic optimization method needs to be given. The optimization goals are:

(a) The difference between the image generated by rendering using the volume depth image and the image generated by direct volume rendering using the original data should be as small as possible. This goal is measured by the root mean square error, and the formula is as follows:

X _obs,i is the color value of the ith pixel of the image drawn using the volume depth image, and X _model,i is the color value of the ith pixel of a reference image. Since each color value is a three-dimensional vector, the calculation of (X _obs,i -X _model,i ) uses the Euler distance formula. The smaller the value, the closer the two images are.

(b) The volume depth image should be as small as possible, and this target is measured using the data compression ratio as follows:

where c is the data compression rate of the global volume depth image, D _i is the data size of the volume depth image generated by the i-th computing node, and D _raw represents the size of the original data.

(c) The generation time of the volume depth image should be as small as possible:

where t is the generation time value of the global volume depth image, and t _i is the time taken by the i-th computing node to generate the volume depth image.

(d) to the above three goals, the present invention uses a weighting method to synthesize it into a single-objective optimization problem, and the formula is as follows:

是数据压缩率的评估函数，两者公式如下：In the formula, k ₁ , k ₂ , and k ₃ are the generation time of the volume depth image, the data compression rate of the volume depth image, and the weight of the rendering quality of the volume depth image, which satisfy k ₁ +k ₂ +k ₃ =1. ψ(t) is the evaluation function of the generation time,

is the evaluation function of the data compression rate, and the two formulas are as follows:

In the formula, α and β are the two proportional division points of the generation time and the compression rate, respectively. The values are in the (0,1) interval and are used for regulation. The meaning is that the generation time is less than α and the compression rate is less than β. parameter group. If the control value is exceeded, an exponential function penalty will be given.

超级片段的合并阈值δ等。本发明使用粒子群算法，进行寻优，如图4所示，其过程包括：There are many factors that affect the above goals, including computing platform characteristics, network characteristics, and raw data characteristics. Among them, the parameters that can be set by the algorithm include: the size of the depth image to be generated, and the sampling interval is recorded as

Merge Threshold δ for Super Fragments, etc. The present invention uses particle swarm algorithm to search for optimization, as shown in Figure 4, the process includes:

(a) Randomly generate J points, each point includes a set of position information and velocity information, the position information is a set of volume depth image parameter groups, the parameter group is randomly generated in the parameter space, and the velocity value is (-1,1) is randomly generated in the parameter space of ;

(b) According to the parameter group of each point, generate a volume depth image, obtain the generation time t and the compression rate c, use the generated volume depth image data to draw an image, and compare with the reference image to calculate the image quality q;

(c) Bring the evaluation value e into the evaluation function E according to t, c, q to calculate the evaluation value e, and update the historical optimal solution and the global optimal solution of the point according to e;

(d) According to the historical optimal solution and the global optimal solution, substitute the following formulas to update the speed and position information of each point:

V _i,t+1 =w*V _i,t +c ₁ *r ₁ *(P _B,i -X _i,t )+c ₂ *r ₂ *( _GB -X _i,t )

X _i,t+1 =X _i,t +r*V _i,t+1 ;

In the formula, Vi _,t+1 and Xi _,t+1 respectively represent the speed and the new position value of the i-th point t+1 round, which are obtained from _Vi,t and Xi _,t of the previous round. c ₁ and c ₂ are learning factors, representing the maximum step size to push each point to the historical optimal solution for that point and the global optimal solution for the point set, respectively. When c ₂ is large, the entire point set will move towards the global optimal solution faster, and vice versa. PB _,i represents the historical optimal solution of the i-th point, and _GB represents the global optimal solution. The parameter w is the inertia weight, a coefficient that maintains the previous velocity, so that the point has an inertia that maintains the original direction of motion. Parameters r ₁ and r ₂ are uniformly distributed random numbers between [0,1], adding random perturbations to the algorithm. The constant r is the constraint factor, a weight for speed, usually set to 1.

(e) Determine whether the iteration ends or not by judging whether the Euler distance between more than half of the points in the point set and the global optimal solution is less than a certain threshold ε. If it does not meet the standard, go back to step (b), otherwise end the optimization, and the global optimal solution is the optimization result.

(4) In the subsequent simulation calculation, the volume depth image is generated in situ according to the optimal parameter group.

Embodiment: The above algorithm is implemented on a supercomputing platform. In the test example, the computing node contains 32 nodes, each node contains 16 Intel(R) Xeon(R) E5620 CPUs, the main frequency is 2.40GHz, the single-node memory is 22G, and the nodes use 1G Infiniband network connection. The draw node uses 2 Intel Core i7-4-core CPUs, 2 Quadro 4000 graphics cards, and 32GB of RAM.

The calculation example uses the direct numerical simulation method (Direct Numerical Simulation, DNS) to simulate the 3-dimensional incompressible fluid isotropic turbulent phenomenon, the grid number is 1024 ³ , and the time step is 1024. Figure 5 is the effect picture of using this method to draw the velocity derivation λ ₂ in DNS, which can show the position of the vortex in the flow field. The rendering effect of this method is very different from the direct volume rendering rendering effect, and there is no obvious difference to the naked eye, but the compression rate after in-situ processing can reach more than 10 times.

Those of ordinary skill in the art can understand that the above are only preferred examples of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art can still Modifications are made to the technical solutions described in the foregoing examples, or equivalent replacements are made to some of the technical features. All modifications and equivalent replacements made within the spirit and principle of the invention shall be included within the protection scope of the invention.

Claims (5)

1. An in-situ volume rendering method, comprising the steps of:

(1) and (3) generating a body depth image: processing data of a calculation result at an in-situ calculation node of large-scale scientific calculation in the following processing mode:

(1.1) emitting a ray from the viewpoint O to each pixel point of a rendering region (W, H), and calculating the intersection point of each ray and a volume data bounding box, wherein W and H respectively represent the width and the height of the rendering region;

(1.2) sampling at equal intervals along the ray from the incident point to the emergent point of the bounding box, wherein the sampling interval is recorded as

Interpolating from peripheral data by using a trilinear interpolation method to obtain a sampling value;

(1.3) obtaining color values c and opacity values alpha of the sampling points by using a transfer function F;

(1.4) when the color similarity of adjacent sampling points on the same ray is smaller than a specific threshold value delta, merging the adjacent sampling points, wherein the merged sampling point set is called a super segment, the attribute of the super segment comprises a starting position, an ending position, a color value and transparency, the combination of all the super segments is called a body depth image and contains depth information and color information of data body drawing when observed from a specific viewpoint;

(2) and (3) volume depth image drawing: and (3) transmitting the volume depth image obtained in the step (1.4) to a drawing node for drawing, wherein the drawing method of the volume depth image comprises the following steps:

(2.1) expanding each super segment into a frustum by using the positions of the pixel points and the positions of the viewpoints during generation;

(2.2) sequencing all the frustum cones according to the depth according to the current drawing viewpoint;

(2.3) directly drawing all the frustum bodies after sequencing, and correcting the transparency of each frustum body during drawing: the transparency is determined according to the relationship between the length of the frustum and the length of the line segment through which the current line of sight passes, using the following formula:

wherein eta is the transparency value of the frustum, s is the length of the frustum, s 'is the length of the line segment of the current sight line passing through the frustum, and eta' is the transparency value after correction;

(3) automatic optimization of body depth image operation parameters

Depth of bodyThe key operation parameter combination in the process of generating the degree image is

Optimizing the data;

(3.1) selecting a group of parameter groups in the parameter space to generate a body depth image, and obtaining corresponding body depth image generation time t and body depth image compression ratio c;

(3.2) drawing the body depth image by using the body depth image to obtain a final image, and calculating the quality q of the final image;

(3.3) substituting the t, c and q values into an evaluation function to obtain the evaluation values of the group of parameters, and updating the global optimal parameter group according to the evaluation values, wherein the evaluation function formula is as follows:

in the formula k₁，k₂，k₃Weights for the generation time of the body depth image, the data compression rate of the body depth image and the rendering quality of the body depth image respectively satisfy k₁+k₂+k₃1, ψ (t) is an evaluation function of the generation time,

is an evaluation function of data compression ratio, and the two formulas are as follows:

in the formula, alpha and beta are two proportional division points of generation time and compression ratio respectively, and values are in a (0,1) interval for regulation and control;

(3.4) finishing the optimization according to the termination condition to obtain an optimal parameter set; otherwise, returning to (3.1);

(4) in the subsequent simulation calculation, the body depth image is generated based on the set of optimum parameter sets.

2. The in-situ volume rendering method according to claim 1, wherein the volume depth image generation time t in (3.1) is obtained by the following formula:

wherein t is_iAnd N is the number of the calculation nodes when the volume depth image is generated for the ith calculation node.

3. The in-situ volume rendering method according to claim 1, wherein the volume depth image compression ratio c in (3.1) is obtained by the following formula:

in the formula, D_iData size, D, of the body depth image generated for the ith compute node_rawRepresenting the size of the original data.

4. The in-situ volume rendering method according to claim 1, wherein the quality q of the image in step (3.2) is obtained by the following formula:

n is the number of image pixels, X_obs，iIs the color value of the ith pixel of an image rendered using a volume depth image, X_model，iIs the color value of the ith pixel of a reference image; due to each colorThe color value is a three-dimensional vector, (X)_obs，i-X_model，i) The euler distance formula is used for the calculation of (a), and the smaller the value, the closer the two images are.

5. The in-situ volume rendering method according to claim 1, wherein the step (3) is optimized by using a particle swarm optimization, and comprises the steps of:

(a) randomly generating J points, wherein each point comprises a group of position information and speed information, the position information is a group of body depth image parameter groups, the parameter groups are randomly generated in a parameter space, and the speed values are randomly generated in a parameter space of (-1, 1);

(b) generating a body depth image according to the parameter set of each point, obtaining generation time t and a compression ratio c, drawing an image by using the generated body depth image data, and comparing the image with a reference image to obtain image quality q;

(c) substituting the evaluation function E into the evaluation functions E according to the t, c and q to obtain an evaluation value E, and updating the historical optimal solution and the global optimal solution of the point according to the E;

(d) updating the speed and position information of each point according to the historical optimal solution and the global optimal solution;

(e) judging whether the Euler distance of more than half of points in the point set and the global optimal solution is less than a certain threshold value epsilon to determine whether the iteration is ended or not; and (c) returning to the step (b) if the optimal solution does not meet the standard, otherwise, finishing the optimization, and obtaining the global optimal solution, namely the optimization result.

Images