CN118200634A - A three-dimensional point cloud transmission method, transmission device, receiving method and receiving device - Google Patents

Abstract

The present invention relates to the field of point cloud technologies, and in particular, to a method and an apparatus for receiving and transmitting a three-dimensional point cloud. A method for transmitting a three-dimensional point cloud, the method comprising: layering, namely dividing the point cloud data into different detail levels according to the detail levels, and generating indication information of each detail level; the encoding step, respectively pre-encoding the point cloud data of the component tracks of each detail level to generate a media code stream corresponding to each detail level; and packaging, namely packaging the indication information of each detail level and the media code stream corresponding to each detail level to generate media code stream information. According to the method and the device, proper detail levels can be selected according to different application scenes, then the three-dimensional point cloud is divided into the detail levels of corresponding levels according to the detail levels, and finally indication information of each detail level and a media code stream are encoded, packaged and transmitted. The transmission method of the three-dimensional point cloud not only can reduce the transmitted data quantity, but also can ensure the playing experience of a user.

Description

A three-dimensional point cloud transmission method, transmission device, receiving method and receiving device

Technical Field

The present invention relates to the field of point cloud technology, and in particular to a method and device for receiving and sending three-dimensional point clouds.

Background technique

With the development of 3D vision and media technology, users have higher and higher requirements for the experience of media consumption, which is reflected in the visual quality of the media and the smoothness of playback. Therefore, it provides users with an immersive experience that is not restricted by time, space or reality conditions, high-quality visual media transmission, and smooth playback experience. As the visual carrier of 3D media, 3D point cloud is an unordered discrete point set distributed in 3D space that expresses geometric structure characteristics and attribute presentation characteristics. The geometric information of 3D point cloud is a set of 3D coordinate information of each point on the surface of the scanned object, and the attribute information includes texture, material, normal vector, reflection intensity, etc. As the main carrier of immersive multimedia scene expression information, 3D point cloud can be used to render and present accurate stereo models in real time, and truly describe static and dynamic objects and scenes. There are significant differences between the generation, packaging, transmission, processing and presentation of immersive media data with point cloud as media data and traditional media data, and immersive media data is more complex and diversified.

In the prior art, in the process of encapsulating and transmitting point cloud data of a three-dimensional point cloud, the complete point cloud detail data is usually encapsulated and transmitted. This method of encapsulating and transmitting the complete point cloud detail data will increase the redundancy of the data and is not conducive to improving the resource utilization of the encapsulated and transmitted data.

Summary of the invention

In view of the problem of low resource utilization in the current three-dimensional point cloud transmission, the present invention provides a method and device for sending and receiving three-dimensional point clouds.

The technical solution adopted by the present invention to solve the technical problem is:

A method for transmitting a three-dimensional point cloud, the method comprising:

A stratification step, dividing the point cloud data into different detail levels according to the detail level, and generating indication information of each detail level;

In the encoding step, the point cloud data of the component tracks at each detail level are pre-encoded respectively to generate a media stream corresponding to each detail level;

In the step of encapsulating, the indication information of each detail level and the media stream corresponding to each detail level are encapsulated to generate media stream information.

Preferably, the indication information of the detail level includes any one or more of the following:

Indicates whether the current component track is a base component track, the ID of the current component track, the reference level of detail of the current component track, and the reference ID of the current component track.

Preferably, when the component track is located in a point cloud geometry component track of type 'apcc', the component track follows the corresponding geometric reference relationship; or

When a component track is located in a point cloud attribute component track of type 'apcc', the component track follows the corresponding same attribute reference relationship; or

When a component track is located in a point cloud piece base track of type 'apeb' or 'apcb', the point cloud piece track samples associated with the sample of the component track follow the corresponding same-component reference relationship.

Preferably, the method for determining the level of detail comprises:

Determine the level of detail based on the spatial scale of the 3D point cloud; or

Setting different levels of detail according to the user perspective and/or position of the user terminal; or

Determines the detail level based on current network conditions.

A method for receiving a three-dimensional point cloud, the method comprising:

The step of decapsulating is to decapsulate the received media code stream information to obtain the media code stream corresponding to each detail level and the indication information of each detail level;

A decoding step, decoding the media stream information according to the indication information of each detail level to obtain point cloud data of the component track at each detail level;

The rendering step renders the decoded point cloud data to obtain a three-dimensional point cloud image/video.

Preferably, the indication information of the detail level includes any one or more of the following:

Indicates whether the current component track is a base component track, the ID of the current component track, the reference level of detail of the current component track, and the reference ID of the current component track.

A three-dimensional point cloud transmission device, the device comprising:

A hierarchical module is used to divide the point cloud data into different detail levels according to the detail level and generate indication information of each detail level;

An encoding module, used to pre-encode the point cloud data of the component tracks at each detail level respectively, and generate a media code stream corresponding to each detail level;

The encapsulation module is used to encapsulate the indication information of each detail level and the media code stream corresponding to each detail level to generate media code stream information.

Preferably, the indication information of the detail level includes any one or more of the following:

Indicates whether the current component track is a base component track, the ID of the current component track, the reference level of detail of the current component track, and the reference ID of the current component track.

Preferably, when the component track is located in a point cloud geometry component track of type 'apcc', the component track follows the corresponding geometric reference relationship; or

When a component track is located in a point cloud attribute component track of type 'apcc', the component track follows the corresponding same attribute reference relationship; or

When a component track is located in a point cloud piece base track of type 'apeb' or 'apcb', the point cloud piece track samples associated with the sample of the component track follow the corresponding same-component reference relationship.

Preferably, the method for determining the level of detail comprises:

Determine the level of detail based on the spatial scale of the 3D point cloud; or

Setting different levels of detail according to the user perspective and/or position of the user terminal; or

Determines the detail level based on current network conditions.

A three-dimensional point cloud receiving device, the device comprising:

A decapsulation module, used to decapsulate the received media code stream information to obtain the media code stream corresponding to each detail level and the indication information of each detail level;

A decoding module, used to decode the media stream information according to the indication information of each detail level to obtain point cloud data of the component track at each detail level;

The rendering module is used to render the decoded point cloud data to obtain a three-dimensional point cloud image/video.

Preferably, the indication information of the detail level includes any one or more of the following:

Indicates whether the current component track is a base component track, the ID of the current component track, the reference level of detail of the current component track, and the reference ID of the current component track.

Beneficial effects of the present invention: The present invention can select appropriate detail levels according to different application scenarios, and then divide the three-dimensional point cloud into detail levels of corresponding levels according to the detail levels, and finally encode, encapsulate and transmit the indication information and media streams of each detail level. This three-dimensional point cloud transmission method can not only reduce the amount of transmitted data, but also ensure the user's playback experience and flexibly adapt to the current bandwidth limitations.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic diagram of a flow chart of a method for transmitting a three-dimensional point cloud according to the present invention;

FIG2 is a schematic diagram of a flow chart of a method for receiving a three-dimensional point cloud according to the present invention;

FIG3 is a schematic structural diagram of a three-dimensional point cloud transmission device according to the present invention;

FIG. 4 is a schematic structural diagram of a three-dimensional point cloud receiving device according to the present invention.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

The present invention can be applied to standard or non-standard point cloud codecs, for example, codecs of the audio video coding standard (AVS).

As the visual carrier of three-dimensional media, three-dimensional point cloud is an unordered discrete point set distributed in three-dimensional space that expresses geometric structure characteristics and attribute presentation characteristics.

Sending method

The present invention provides a method for sending a three-dimensional point cloud, which can be applied to a user end and a server end. The user end is a device worn by a user. When the user end wants to obtain a video or an image, the relevant video or image can be obtained through the server end.

The present invention provides an embodiment of a method for sending a three-dimensional point cloud. As shown in FIG1 , the sending method includes:

A layering step S11, dividing the point cloud data into different detail levels according to the detail level, and generating indication information of each detail level;

The encoding step S12 is to pre-encode the point cloud data of the component tracks at each detail level to generate a media code stream corresponding to each detail level;

In the encapsulation step S13, the indication information of each detail level and the media stream corresponding to each detail level are encapsulated to generate media stream information.

In this embodiment, a suitable level of detail can be selected according to different application scenarios, and then the three-dimensional point cloud can be divided into corresponding levels of detail according to the level of detail, and finally the indication information and media stream of each level of detail can be encoded, packaged and transmitted. This method of transmitting three-dimensional point clouds can not only reduce the amount of transmitted data, but also ensure the user's playback experience and flexibly adapt to the current bandwidth limitations.

In one embodiment, the original point cloud data may be classified by level of detail (LOD), where LOD represents the detail of the point cloud content. As the LOD value decreases, it indicates that the detail of the point cloud content decreases, and the space (or distance) between the points becomes wider. As the LOD value increases, it indicates that the detail of the point cloud content increases, and the space (or distance) between the points becomes narrower. That is, the points in the lowest LOD are sparsely distributed, and the points in the highest LOD are densely distributed.

In point cloud data, the point cloud data corresponding to the lowest level of detail after stratification can be called the basic layer. Selecting an appropriate distance threshold parameter allows the basic layer to describe the basic outline of the original point cloud. The remaining detail levels are called enhancement layers, which describe the detailed information of the original point cloud. All enhancement layers depend on the basic layer and the enhancement layers with larger distance threshold parameters of the previous detail levels. The enhancement layers describe very detailed point clouds. The higher the enhancement layer, the more it depends on the previous enhancement layer. The lower the detail level, the lower the detail level, and the higher the distance threshold parameter. The higher the detail level, the higher the detail level, and the lower the distance threshold parameter.

In one embodiment, the three-dimensional point cloud is composed of point cloud data of different detail levels, each detail level corresponds to point cloud data of a detail level, and the point cloud data of the original point cloud is hierarchically divided according to certain rules to obtain point cloud data of multiple detail levels. Point cloud data is divided into multiple types, such as geometry, occupancy map, feature, attribute, auxiliary, color, texture, material, normal vector, reflection intensity, etc.

In step S11 of the above embodiment, the original point cloud is divided into layers according to the detail level, which is an iterative process. It can be divided into one detail level or multiple detail levels. When the original point cloud is divided into one detail level according to the detail level, the specific process is:

S111, randomly select a point in the original point cloud as a reference point and mark it as visited;

S112, according to a preset first distance threshold parameter, find a point whose geometric distance to the reference point is closest and greater than the first distance threshold parameter, use this point as the current reference point, and mark it as visited;

S113, repeating steps S111 and S112 until no point greater than the first distance threshold parameter and not marked as visited is found;

S114, separating the visited reference points from the original point cloud data as point cloud data of a detail level.

In one embodiment, when the point cloud data needs to be divided into more detail levels LOD corresponding to the detail levels, as the LOD value increases, the details of the point cloud content are enhanced, and the distance between each point becomes narrower, so the distance threshold parameter corresponding to the later detail level is less than the distance threshold parameter corresponding to the previous detail level. For the previous embodiment, it is necessary to continue to execute:

S115, presetting a second distance threshold parameter, where the second threshold distance parameter is smaller than the first distance threshold parameter;

S116, for the points in the original point cloud that are not marked as visited, repeat the above process S111-S115 until all the points in the original point cloud are marked as visited.

In one embodiment, the point clouds of each detail level may be gradually layered according to the geometric distance and/or color attribute of the point cloud. The point clouds of each detail level may be gradually layered according to the geometric distance or color attribute, or the two may be combined for gradual layering. For example, when searching for two points with the same geometric distance from a preset distance threshold parameter, the point closest to the reference point may be further determined according to the color (RGB value or other representation). Similarly, if two points have the same color attribute, the greater the geometric distance, the greater the total distance.

In one embodiment, a method for determining a level of detail includes:

Determine the level of detail based on the spatial scale of the 3D point cloud; or

Setting different levels of detail according to the user perspective and/or position of the user terminal; or

Determines the detail level based on current network conditions.

Specifically, the level of detail can be determined according to the spatial scale of the 3D point cloud. In other words, the user end can freely enlarge or reduce the spatial scale of the point cloud data, similar to the enlargement or reduction of a picture. Changing the spatial scale will not change the amount of data, but after enlarging, more details will be presented in the presented point cloud image or video. Therefore, when the user enlarges the spatial scale, the required level of detail is higher; when the spatial scale is reduced, fewer details will be presented. At this time, lowering the level of detail will not have a great impact on the user's viewing experience, so the level of detail can be lowered to save transmission bandwidth.

In an embodiment in which different levels of detail are set according to the user perspective and/or position of the user terminal, the user terminal is a device worn by the user for viewing the point cloud video/image sent by the server, and the device can select the user perspective and position by moving the head. For point cloud data at the edge of the vision and outside the perspective, the required level of detail is lower; for point cloud data at the center of the perspective, the required level of detail is higher. Determining the level of detail required for each spatial area of the point cloud data under the current perspective and/or position can reduce the bandwidth required for transmitting point cloud data at the edge of the vision and outside the perspective.

In the embodiment of determining the level of detail according to the current network conditions, the user terminal has a higher requirement for the smoothness of playback and a slightly lower requirement for the visual quality of a single frame, such as the use scenario of live broadcast. Therefore, the required level of detail and the corresponding level of detail can be selected according to the current network bandwidth and other network conditions. For example, if the current network bandwidth is limited, a media stream with a small number of enhancement layers or even only the base layer can be selected to ensure smooth playback.

The detail level of the present invention can be determined by the user end sending user request information to the server end, and the detail level can be confirmed according to the user request information. It can also be specified by the server end according to the encoding algorithm. For example, for the above three embodiments of determining the detail level, the detail level is determined according to the spatial scale of the three-dimensional point cloud; and the implementation method of setting different detail levels according to the user perspective and/or position of the user end can be determined by the user end, and the determined detail level is sent to the server end. For the implementation method of determining the detail level according to the current network conditions, it can be determined by the server end according to the encoding algorithm.

In the encoding step S12, the point cloud data of the component tracks at each detail level are pre-encoded respectively to generate a media code stream corresponding to each detail level.

For each level of detail, the corresponding detail layer includes point cloud data of a component track. In this step, the point cloud data is encoded at a suitable bit rate and/or a bit rate corresponding to the level of detail indication information of the detail layer, so that a media stream corresponding to each level of detail can be generated.

In one embodiment, the encoding method includes separate encoding, that is, the base layer and each enhancement layer of the point cloud can be independently encoded. In this case, each detail level corresponds to a media stream; the base layer can be encoded separately to form a media stream of the base layer; and each enhancement layer can be encoded to form a media stream of the encoding layer.

In another embodiment, the encoding method includes combined encoding: first, the base layer is encoded, and then all enhancement layers are encoded one by one. For each enhancement layer, the enhancement layer is spatially combined with the base layer and all enhancement layers with lower detail levels, and then encoded. Combined encoding can improve encoding efficiency and facilitate subsequent packaging and media stream selection.

The present invention does not limit the encoding of each detail level, and the appropriate combination and code rate can be selected by the user.

S13, a packaging step of packaging the indication information of each detail level and the media stream corresponding to each detail level to generate media stream information.

Each level of detail indication information is used to indicate the level of detail of the pre-encoded media stream. Each level of detail indication information may include the level of detail of the component track of the corresponding detail level.

In one embodiment, the level of detail indication information includes any one or more of the following: an indication of whether the current component track is a base component track, an identifier of the current component track, a reference level of detail of the current component track, and a reference identifier of the current component track.

In another embodiment, the detail level indication information may also include any one or more of the following: the number of detail levels, i.e., the number of enhancement layers, the identification of the detail level, the distance threshold parameter of the detail level, the number of component tracks corresponding to the detail level, the identification of the component tracks corresponding to the detail level, the type of the component track group corresponding to the detail level, the number of component track groups corresponding to the detail level, and the identification of the component track group corresponding to the detail level.

In one embodiment, a component track is encapsulated as a media asset, and a component track group consisting of one or more component tracks is encapsulated as a data package; each media asset contains point cloud data contained in a component track selected by a detail level, and each data package contains point cloud data of a component track selected by each point cloud detail level.

A more specific embodiment is used below to introduce a scalable component track information data box ScalabilityComponentTrackGroupInfoBox, which is used to indicate the indication information of component tracks at each detail level. The following lists the syntax example of the scalable component track information data box.

definition

Data box type: 'scif'

Included in: Sample Entry

Is it mandatory: No

Quantity: 0 or 1

The scalable component track information data box ScalabilityComponentTrackGroupInfoBox is used to indicate reference information of the corresponding component track.

When the ScalabilityComponentTrackGroupInfoBox is located in a point cloud geometry component track of type 'apcc', the track corresponding to the ScalabilityComponentTrackGroupInfoBox follows the corresponding geometric reference relationship.

When the ScalabilityComponentTrackGroupInfoBox is located in a point cloud attribute component track of type 'apcc', the track corresponding to the ScalabilityComponentTrackGroupInfoBox follows the corresponding same-attribute reference relationship.

When the ScalabilityComponentTrackGroupInfoBox is located in a point cloud piece base track of type 'apeb' or 'apcb', the point cloud piece track samples associated with the samples corresponding to the ScalabilityComponentTrackGroupInfoBox follow the corresponding same-component reference relationship.

grammar

Semantics

component_base_track_flag indicates whether the current component track is a base component track. When the value is 1, it indicates that the current component track is a base component track, and when the value is 0, it indicates that the current component track is a non-base component track.

current_track_id indicates the ID of the current component track.

reference_level indicates the reference detail level of the current component track. The smaller the value of this field, the higher the reference of this component track to the base component track.

reference_track_id indicates the reference identifier of the current component track.

In the above embodiment, the scalable component track information data box is used as a kind of detail level indication information to indicate the information of the component track at each detail level.

When the Scalable Component Track Information data box is in a point cloud geometry component track of type 'apcc', the data box describes the level of detail of the geometry component track, as well as the geometry component tracks of other point clouds that need to be referenced.

When the scalable component track information data box is in the point cloud attribute component track of type 'apcc', the data box describes the level of detail of the attribute component track, as well as the component tracks of other point cloud attributes that need to be referenced. For example, when the attribute component track is a color component track, the data box describes the level of detail corresponding to the current color attribute information, as well as the color information of the level of detail of other point clouds that need to be referenced.

When the scalable component track information data box is in a point cloud slice base component track of type 'apeb' or 'apcb', the data box describes the level of detail of the point cloud slice track sample associated with the sample of the current component track, as well as the point cloud slice tracks corresponding to the level of detail of other point clouds that need to be referenced.

In the above syntax, for other definitions, extends AVSPCCSampleEntry()('satg') indicates that the data box is extended based on the standard class AVSPCCSampleEntry(), and the data box should be stored in the sample entry.

For the syntax:

if(!component_base_track_flag){

unsigned int(8)reference_level

unsigned int(32)reference_track_id

}

Indicates that if the current detail level is not the basic layer, that is, the current component track is a non-base component track, the detail level indication information indicates the reference detail level required by the current component track, and indicates the reference identifier required by the current component track. The so-called reference identifier refers to the identifier ID of the component track.

The reference detail level indicates the detail level of the component track that the current component track needs to refer to. The reference identifier indicates the identification ID of the component track that the current component track needs to refer to. For example, a three-dimensional point cloud data contains a basic layer and two enhanced layers. For the first enhanced layer, the component_base_track_flag in the detail level indication information of its component track indicates that the current detail level is not the basic layer, then execute: indicate through reference_level that the reference detail level that the component track of the first enhanced layer needs to refer to is the basic layer. Indicate through reference_track_id that the reference identifier that the component track of the first enhanced layer needs to refer to is the track identifier of the basic layer. Because the layering of the first enhanced layer depends on the basic layer, it is necessary to refer to the point cloud data information of the basic layer. By analogy, for the second enhanced layer, the component_base_track_flag in the detail level indication information of its component track indicates that the current detail level is not the basic layer, then execute: indicate through reference_level that the reference detail level that the component track of the second enhanced layer needs to refer to is the first enhanced layer. Then indicate through reference_track_id that the reference identifier that the component track of the second enhanced layer needs to refer to is the track identifier of the first enhanced layer. Because the layering of the second enhancement layer depends on the first enhancement layer of the previous layer, it is necessary to refer to the point cloud data of the first enhancement layer.

This data box describes the dependency relationship between the point cloud data in the base layer and the enhancement layer by defining whether the current component track is the base layer, and the reference detail level and reference detail identifier of the component track that the current component track needs to refer to. This allows the user end and the server end to link the component tracks of the required detail level according to the needs, and encode and encapsulate the media streams therein.

In the above-described encapsulation embodiment, the media stream information is generated by encapsulating the indication information of each detail level and the media stream corresponding to each detail level. This encapsulation method is suitable for encapsulation of multiple component tracks (ie, component track groups).

Here, another embodiment is introduced, in which the indication information of each level of detail and the corresponding media stream are encapsulated separately. This situation is suitable for the case of multi-track encapsulation of slices. The indication information of each level of detail will be placed in the slice base track, and the corresponding media stream will be placed in the corresponding slice track. For example, the geometric information of a detail level is placed in the slice geometry component track, and its attribute information is placed in the slice attribute component track.

Receiving method

A method for receiving a three-dimensional point cloud, as shown in FIG2, includes:

Decapsulation step S21, decapsulating the received media code stream information to obtain the media code stream corresponding to each detail level and indication information of each detail level;

Decoding step S22, decoding the media stream information according to the indication information of each detail level to obtain point cloud data of the component track of each detail level;

In the rendering step S23, the decoded point cloud data is rendered to obtain a three-dimensional point cloud image/video.

The receiving method can receive the indication information for dividing the three-dimensional point cloud into different detail levels, and then decode the point cloud data of the component track corresponding to each detail level according to the indication information. This three-dimensional point cloud transmission method can not only reduce the amount of transmitted data, but also ensure the user's playback experience and flexibly adapt to the current bandwidth limitation.

In one embodiment, the indication information of each detail level obtained by decapsulation may be the ScalabilityComponentTrackGroupInfoBox data box introduced in the sending method, which is used to indicate the indication information of the component track at each detail level. The syntax example and field introduction of the ScalabilityComponentTrackGroupInfoBox data box are the same as those shown in the sending method, and are not repeated here.

In one embodiment, the level of detail indication information includes any one or more of the following: an indication of whether the current component track is a base component track, an identifier of the current component track, a reference level of detail of the current component track, and a reference identifier of the current component track. Specifically:

component_base_track_flag indicates whether the current component track is a base component track. When the value is 1, it indicates that the current component track is a base component track, and when the value is 0, it indicates that the current component track is a non-base component track.

current_track_id indicates the ID of the current component track.

reference_level indicates the reference detail level of the current component track. The smaller the value of this field, the higher the reference of this component track to the base component track.

reference_track_id indicates the reference identifier of the current component track.

In another embodiment, the detail level indication information may also include any one or more of the following: the number of detail levels, i.e., the number of enhancement layers, the identification of the detail level, the distance threshold parameter of the detail level, the number of component tracks corresponding to the detail level, the identification of the component tracks corresponding to the detail level, the type of the component track group corresponding to the detail level, the number of component track groups corresponding to the detail level, and the identification of the component track group corresponding to the detail level.

In one embodiment, when the scalable component track information data box ScalabilityComponentTrackGroupInfoBox is located in a point cloud geometry component track of the 'apcc' type, the track corresponding to the scalable component track information data box follows the corresponding geometric reference relationship.

In one embodiment, when the scalable component track information data box ScalabilityComponentTrackGroupInfoBox is located in a point cloud attribute component track of the 'apcc' type, the track corresponding to the scalable component track information data box follows the corresponding same-attribute reference relationship.

In one embodiment, when the scalable component track information data box ScalabilityComponentTrackGroupInfoBox is located in a point cloud piece base track of type 'apeb' or 'apcb', the point cloud piece track samples associated with the samples corresponding to the scalable component track information data box follow the corresponding same-component reference relationship.

In one embodiment, the level of detail in the receiving method is received from the level of detail sent by the sending end (the user end or the server end as the case may be), and the method for determining the level of detail includes:

Determine the level of detail based on the spatial scale of the 3D point cloud; or

Setting different levels of detail according to the user perspective and/or position of the user terminal; or

Determines the detail level based on current network conditions.

Transmission device

A transmission device for a three-dimensional point cloud, as shown in FIG3, comprises:

A layering module 11, used to divide the point cloud data into different detail levels according to the detail level, and generate indication information of each detail level;

The encoding module 12 is used to pre-encode the point cloud data of the component track at each detail level respectively to generate a media code stream corresponding to each detail level;

The encapsulation module 13 is used to encapsulate the indication information of each detail level and the media stream corresponding to each detail level to generate media stream information.

The transmission device can select the appropriate level of detail according to different application scenarios, and then divide the 3D point cloud into corresponding levels of detail according to the level of detail, and finally encode, package and transmit the indication information and media stream of each level of detail. This 3D point cloud transmission device can not only reduce the amount of transmitted data, but also ensure the user's playback experience and flexibly adapt to the current bandwidth limitations.

In one embodiment, the indication information of each detail level generated by the layering module 11 includes any one or more of the following: an indication of whether the current component track is a base component track, an identifier of the current component track, a reference detail level of the current component track, and a reference identifier of the current component track. The indication information of each detail level may be the ScalabilityComponentTrackGroupInfoBox, a scalable component track information data box introduced in the corresponding sending method, which is used to indicate the indication information of the component track at each detail level. The syntax example and field introduction of the scalable component track information data box are the same as those shown in the sending method, and are not repeated here.

In another embodiment, the detail level indication information may also include any one or more of the following: the number of detail levels, i.e., the number of enhancement layers, the identification of the detail level, the distance threshold parameter of the detail level, the number of component tracks corresponding to the detail level, the identification of the component tracks corresponding to the detail level, the type of the component track group corresponding to the detail level, the number of component track groups corresponding to the detail level, and the identification of the component track group corresponding to the detail level.

In another embodiment, when the component track is located in a point cloud geometry component track of type 'apcc', the component track follows the corresponding geometric reference relationship; or

When a component track is located in a point cloud attribute component track of type 'apcc', the component track follows the corresponding same attribute reference relationship; or

When a component track is located in a point cloud piece base track of type 'apeb' or 'apcb', the point cloud piece track samples associated with the component track samples follow the corresponding same-component reference relationship.

In another embodiment, the method for determining the level of detail in the layering module 11 includes:

Determine the level of detail based on the spatial scale of the 3D point cloud; or

Setting different levels of detail according to the user perspective and/or position of the user terminal; or

Determines the detail level based on current network conditions.

The specific methods of each embodiment of determining the detail level can be found in the content introduced in the corresponding sending method, and will not be repeated here.

Receiving device

A three-dimensional point cloud receiving device, as shown in FIG4, includes:

The decapsulation module 21 is used to decapsulate the received media code stream information to obtain the media code stream corresponding to each detail level and the indication information of each detail level;

A decoding module 22, configured to decode the media stream information according to the indication information of each detail level, and obtain point cloud data of the component track of each detail level;

The rendering module 23 is used to render the decoded point cloud data to obtain a three-dimensional point cloud image/video.

The receiving device can receive the indication information for dividing the three-dimensional point cloud into different detail levels, and then decode the point cloud data of the component track corresponding to each detail level according to the indication information. This three-dimensional point cloud transmission method can not only reduce the amount of transmitted data, but also ensure the user's playback experience and flexibly adapt to the current bandwidth limitation.

In one embodiment, the detail level indication information received by the receiving device includes any one or more of the following: an indication of whether the current component track is a base component track, an identifier of the current component track, a reference detail level of the current component track, and a reference identifier of the current component track.

In one embodiment, the indication information of each detail level obtained by decapsulating the receiving device may be the ScalabilityComponentTrackGroupInfoBox data box introduced in the sending method, which is used to indicate the indication information of the component track at each detail level. The syntax example and field introduction of the ScalabilityComponentTrackGroupInfoBox data box are the same as those shown in the sending method, and are not repeated here.

In another embodiment, the detail level indication information may also include any one or more of the following: the number of detail levels, i.e., the number of enhancement layers, the identification of the detail level, the distance threshold parameter of the detail level, the number of component tracks corresponding to the detail level, the identification of the component tracks corresponding to the detail level, the type of the component track group corresponding to the detail level, the number of component track groups corresponding to the detail level, and the identification of the component track group corresponding to the detail level.

In one embodiment, when a component track is located in a point cloud geometry component track of type 'apcc', the component track follows the corresponding geometric reference relationship; or

When a component track is located in a point cloud attribute component track of type 'apcc', the component track follows the corresponding same attribute reference relationship; or

When a component track is located in a point cloud piece base track of type 'apeb' or 'apcb', the point cloud piece track samples associated with the sample of the component track follow the corresponding same-component reference relationship.

In one embodiment, the detail level in the receiving device is received from the detail level sent by the sending end (the user end or the server end as the case may be), and the method for determining the detail level includes:

Determine the level of detail based on the spatial scale of the 3D point cloud; or

Setting different levels of detail according to the user perspective and/or position of the user terminal; or

Determines the detail level based on current network conditions.

The above description is only a preferred embodiment of the present invention, and does not limit the implementation mode and protection scope of the present invention. For those skilled in the art, it should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the description and illustrations of the present invention should be included in the protection scope of the present invention.

Claims (12)

1. A method for transmitting a three-dimensional point cloud, characterized in that the method comprises: The step of stratification is to divide the point cloud data into different detail levels according to the detail level and generate the indication information of each detail level; In the encoding step, the point cloud data of the component tracks at each detail level are pre-encoded respectively to generate a media stream corresponding to each detail level; In the step of encapsulating, the indication information of each detail level and the media stream corresponding to each detail level are encapsulated to generate media stream information. 2. The method for transmitting a three-dimensional point cloud according to claim 1, wherein the information indicating the level of detail comprises any one or more of the following: Indicates whether the current component track is a base component track, the ID of the current component track, the reference level of detail of the current component track, and the reference ID of the current component track. 3. The three-dimensional point cloud transmission method according to claim 1 or 2, characterized in that: When a component track is located in a point cloud geometry component track of type 'apcc', the component track follows the corresponding geometry reference relationship; or When a component track is located in a point cloud attribute component track of type 'apcc', the component track follows the corresponding same attribute reference relationship; or When a component track is located in a point cloud piece base track of type 'apeb' or 'apcb', the point cloud piece track samples associated with the sample of the component track follow the corresponding same-component reference relationship. 4. The method for transmitting a three-dimensional point cloud according to claim 1 or 2, wherein the method for determining the level of detail comprises: Determine the level of detail based on the spatial scale of the 3D point cloud; or Setting different levels of detail according to the user perspective and/or position of the user terminal; or Determines the detail level based on current network conditions. 5. A method for receiving a three-dimensional point cloud, characterized in that the method comprises: The step of decapsulating is to decapsulate the received media code stream information to obtain the media code stream corresponding to each detail level and the indication information of each detail level; A decoding step, decoding the media stream information according to the indication information of each detail level to obtain point cloud data of the component track at each detail level; The rendering step renders the decoded point cloud data to obtain a three-dimensional point cloud image/video. 6. The method for receiving a three-dimensional point cloud according to claim 5, wherein the information indicating the level of detail comprises any one or more of the following: Indicates whether the current component track is a base component track, the ID of the current component track, the reference level of detail of the current component track, and the reference ID of the current component track. 7. A three-dimensional point cloud transmission device, characterized in that the device comprises: A hierarchical module is used to divide the point cloud data into different detail levels according to the detail level and generate indication information of each detail level; An encoding module, used to pre-encode the point cloud data of the component tracks at each detail level respectively, and generate a media code stream corresponding to each detail level; The encapsulation module is used to encapsulate the indication information of each detail level and the media code stream corresponding to each detail level to generate media code stream information. 8. The three-dimensional point cloud transmission device according to claim 7, wherein the detail level indication information includes any one or more of the following: Indicates whether the current component track is a base component track, the ID of the current component track, the reference level of detail of the current component track, and the reference ID of the current component track. 9. The three-dimensional point cloud transmission device according to claim 7 or 8, characterized in that: When a component track is located in a point cloud geometry component track of type 'apcc', the component track follows the corresponding geometry reference relationship; or When a component track is located in a point cloud attribute component track of type 'apcc', the component track follows the corresponding same attribute reference relationship; or When a component track is located in a point cloud piece base track of type 'apeb' or 'apcb', the point cloud piece track samples associated with the sample of the component track follow the corresponding same-component reference relationship. 10. The three-dimensional point cloud transmission device according to claim 7 or 8, characterized in that the method for determining the level of detail comprises: Determine the level of detail based on the spatial scale of the 3D point cloud; or Setting different levels of detail according to the user perspective and/or position of the user terminal; or Determines the detail level based on current network conditions. 11. A three-dimensional point cloud receiving device, characterized in that the device comprises: A decapsulation module, used to decapsulate the received media code stream information to obtain the media code stream corresponding to each detail level and the indication information of each detail level; A decoding module, used to decode the media stream information according to the indication information of each detail level to obtain point cloud data of the component track at each detail level; The rendering module is used to render the decoded point cloud data to obtain a three-dimensional point cloud image/video. 12. The device for receiving a three-dimensional point cloud according to claim 11, wherein the information indicating the level of detail comprises any one or more of the following: Indicates whether the current component track is a base component track, the ID of the current component track, the reference level of detail of the current component track, and the reference ID of the current component track.