CN113450127B

CN113450127B - Information display method, information display device, computer equipment and storage medium

Info

Publication number: CN113450127B
Application number: CN202010225927.0A
Authority: CN
Inventors: 李邦鹏; 张梦一; 符永顺; 贺旭; 陈功
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2020-03-26
Filing date: 2020-03-26
Publication date: 2025-07-29
Anticipated expiration: 2040-03-26
Also published as: CN113450127A

Abstract

The present application relates to an information display method, apparatus, computer device, and storage medium, and relates to the field of Internet application technology. The method comprises: obtaining actual display results of each displayable information; obtaining predicted display results of at least two sub-sorting models corresponding to each displayable information; obtaining calibration coefficients of the at least two sub-sorting models corresponding to each displayable information based on the actual display results of each displayable information and the predicted display results of the at least two sub-sorting models corresponding to each displayable information; and subsequently displaying each displayable information according to the displayable information sorting model and the calibration coefficients of the at least two sub-sorting models corresponding to each displayable information. By displaying each displayable information based on the sorting results of the calibrated displayable information sorting model, the accuracy of the displayable information sorting model is improved, thereby improving the display effect of each displayable information.

Description

Information display method, information display device, computer equipment and storage medium

Technical Field

The embodiment of the application relates to the technical field of Internet application, in particular to an information display method, an information display device, computer equipment and a storage medium.

Background

With the continuous development of network and computer technology, online advertisements become an effective means for media hosts to convert user traffic into cash earnings, and in order to improve the accuracy of online advertisement delivery, after each online advertisement is sequenced through a sequencing model, the online advertisements are selected and displayed according to sequencing results.

In the related art, during training of the ranking model, individual models that make up the information ranking model are typically calibrated by using a validation dataset of model evaluations. For example, the model training device may obtain a training data set and a validation data set, train the ranking model from the training data set, and then input the validation data set into the ranking model to calibrate the ranking model.

However, the scheme shown in the related art is only suitable for calibrating a single-stage sorting model through a pre-acquired verification data set in the training process, is not suitable for a multi-stage sorting model, and cannot change the model in the subsequent model application process, so that the accuracy of the sorting model is poor.

Disclosure of Invention

The embodiment of the application provides an information display method, an information display device, computer equipment and a storage medium, which can improve the accuracy of sorting and displaying displayable information, and the technical scheme is as follows:

In one aspect, there is provided an information display method, the method including:

The method comprises the steps of obtaining actual display results of each piece of displayable information, wherein the displayable information is obtained by predicting the prediction probability of receiving the operation of a specified user through a displayable information sorting model and selecting the information to be displayed according to the prediction probability obtained by prediction;

obtaining prediction display results of the at least two sub-ranking models, which correspond to the respective displayable information, wherein the prediction display results are used for indicating the prediction probability of the at least two sub-ranking models for predicting the respective displayable information and receiving the operation of the specified user;

Based on the actual display result of each displayable information and the prediction display result of each displayable information corresponding to the at least two sub-ordering models, acquiring the calibration coefficients of each displayable information corresponding to the at least two sub-ordering models;

based on the calibration coefficients of the at least two sub-ranking models, the prediction probability of each exposable message by the at least two sub-ranking models is modified.

In another aspect, there is provided an information display apparatus, the apparatus comprising:

The system comprises a first acquisition module, a second acquisition module, a third acquisition module and a display module, wherein the first acquisition module is used for acquiring the actual display result of each piece of displayable information, the displayable information is the information which is predicted to receive the predicted probability of the operation of a specified user through a displayable information sequencing model and is selected to be displayed according to the predicted probability obtained by the prediction;

The second acquisition module is used for acquiring prediction display results of the at least two sub-ranking models, corresponding to the respective displayable information, and the prediction display results are used for indicating the prediction probability of the at least two sub-ranking models for predicting and receiving the operation of the specified user to the respective displayable information;

The third acquisition module is used for acquiring calibration coefficients of the at least two sub-ordering models corresponding to the respective displayable information based on the actual display results of the respective displayable information and the predicted display results of the at least two sub-ordering models corresponding to the respective displayable information;

And the display module is used for correcting the prediction probability of each piece of displayable information of the at least two sub-ordering models based on the calibration coefficients of the at least two sub-ordering models.

Optionally, the at least two sub-ranking models include a roughing ranking model and a refining ranking model cascaded after the roughing ranking model;

The second acquisition module includes:

The first acquisition sub-module is used for acquiring calibration coefficients of the selected sorting models corresponding to the respective displayable information based on the actual display results of the respective displayable information and the predicted display results of the selected sorting models corresponding to the respective displayable information;

and the second acquisition sub-module is used for acquiring the calibration coefficients of the rough sorting model corresponding to the displayable information respectively based on the predicted display results of the fine sorting model corresponding to the displayable information respectively and the predicted display results of the rough sorting model corresponding to the displayable information respectively.

Optionally, the first obtaining sub-module includes:

the first acquisition unit is used for acquiring a first calibration coefficient initial value of the carefully chosen sorting model based on the actual display result of each piece of displayable information and the predicted display result of the carefully chosen sorting model, wherein each piece of displayable information corresponds to the calibration coefficient target value of the carefully chosen sorting model;

And the second acquisition unit is used for acquiring the calibration coefficients of the selected sorting models, which are respectively corresponding to the displayable information, based on the first calibration coefficient initial value and the calibration coefficient target values of the selected sorting models, which are respectively corresponding to the displayable information.

Optionally, the first obtaining unit includes:

The first acquisition subunit is used for acquiring a first calibration coefficient initial value of the carefully chosen sorting model based on the actual display result of each piece of displayable information in a first time period and the prediction display result of each piece of displayable information corresponding to the carefully chosen sorting model in the first time period;

And the second acquisition subunit is used for acquiring the calibration coefficient target values of the selected sorting models corresponding to the respective displayable information respectively based on the actual display results of the respective displayable information in the second time period.

Optionally, the actual display result comprises an actual exposure number, an actual click number and an actual conversion number, the predicted display result comprises a predicted click rate and a predicted conversion rate, and the calibration coefficient comprises a click rate calibration coefficient and a conversion rate calibration coefficient;

The first acquisition subunit is configured to:

Obtaining the sum of the actual click numbers of the displayable information in the first time period and the sum of the actual conversion numbers of the displayable information in the first time period;

based on the actual exposure number of each displayable information in the first time period and the estimated click rate and the estimated conversion rate of each displayable information corresponding to the carefully chosen sorting model in the first time period, obtaining the sum of the first estimated click numbers of each displayable information in the first time period and the sum of the first estimated conversion numbers of each displayable information in the first time period, wherein the corresponding carefully chosen sorting model is obtained;

Acquiring a first click rate calibration coefficient initial value of the carefully selected sorting model based on the sum of the actual click numbers of each displayable information in the first time period and the sum of the first estimated click numbers;

And obtaining a first conversion rate calibration coefficient initial value of the carefully chosen sorting model based on the sum of the actual conversion numbers of the various displayable information in the first time period and the sum of the first estimated conversion numbers.

The second acquisition subunit is configured to:

Acquiring the actual click number of each piece of displayable information in the second time period and the actual conversion number of each piece of displayable information in the second time period;

based on the actual exposure number of each piece of displayable information in the second time period, and the estimated click rate and the estimated conversion rate of each piece of displayable information corresponding to the carefully chosen sorting model in the second time period, acquiring the estimated click number and the estimated conversion number of each piece of displayable information corresponding to the carefully chosen sorting model in the second time period;

Based on the actual click number of each displayable information in the second time period and the estimated click number of each displayable information in the second time period corresponding to the carefully chosen sorting model, acquiring a click rate calibration coefficient target value of each displayable information corresponding to the carefully chosen sorting model respectively;

And acquiring conversion rate calibration coefficient target values of the respective displayable information corresponding to the carefully chosen sorting model based on the actual conversion numbers of the respective displayable information in the second time period and the estimated conversion numbers of the respective displayable information in the second time period, which correspond to the carefully chosen sorting model.

Optionally, the second obtaining unit is configured to perform weighted summation on the first calibration coefficient initial value and the calibration coefficient target value of the selected sorting model corresponding to each piece of displayable information, so as to obtain the calibration coefficient of the selected sorting model corresponding to each piece of displayable information.

Optionally, the second obtaining sub-module includes:

the third obtaining unit is used for obtaining a second calibration coefficient initial value of the rough sorting model based on the prediction display result of the fine sorting model corresponding to each piece of displayable information and the prediction display result of the rough sorting model corresponding to each piece of displayable information, and obtaining a calibration coefficient target value of the initial sorting model corresponding to each piece of displayable information;

And the fourth acquisition unit is used for acquiring the calibration coefficients of the rough sorting model, which are respectively corresponding to the displayable information, based on the initial values of the second calibration coefficients and the calibration coefficient target values of the initial sorting model, which are respectively corresponding to the displayable information.

Optionally, the third obtaining unit includes:

The third obtaining subunit is configured to obtain a second calibration coefficient initial value of the rough sorting model based on the predicted display result of each displayable information corresponding to the carefully sorting model in a third time period and the predicted display result of each displayable information corresponding to the rough sorting model in the third time period;

the fourth obtaining subunit is configured to obtain, based on the predicted display results of the respective displayable information corresponding to the selected sorting model in a fourth time period, calibration coefficient target values of the respective displayable information corresponding to the rough sorting model.

Optionally, the predicted display result comprises a predicted click rate and a predicted conversion rate, and the calibration coefficient comprises a click rate calibration coefficient and a conversion rate calibration coefficient;

The third obtaining subunit is configured to obtain, based on the actual exposure number of each displayable information in the third time period and the estimated click rate and the estimated conversion rate of each displayable information corresponding to the carefully-selected ranking model in the third time period, a sum of the second estimated click numbers of each displayable information in the third time period and a sum of the second estimated conversion numbers of each displayable information in the third time period, where the corresponding carefully-selected ranking model;

based on the actual exposure number of each piece of displayable information in the third time period, and the estimated click rate and the estimated conversion rate of each piece of displayable information corresponding to the rough sorting model in the third time period, respectively, obtaining the sum of the third estimated click number of each piece of displayable information in the third time period and the sum of the third estimated conversion number of each piece of displayable information in the third time period, wherein the corresponding rough sorting model;

acquiring a second click rate calibration coefficient initial value of the rough sorting model based on the sum of second estimated click times of each displayable information in the third time period and the sum of third estimated click times of each displayable information in the third time period;

And obtaining a second conversion rate calibration coefficient initial value of the roughing sequencing model based on the sum of the second estimated conversion numbers of the displayable information in the third time period and the sum of the third estimated conversion numbers of the displayable information in the third time period.

The fourth obtaining subunit is configured to obtain, based on the actual exposure number of each displayable information in the fourth time period and the estimated click rate and the estimated conversion rate of each displayable information corresponding to the carefully chosen sorting model in the fourth time period, the estimated click number and the estimated conversion rate of each displayable information in the fourth time period corresponding to the carefully chosen sorting model;

based on the actual exposure number of each piece of displayable information in the fourth time period, and the estimated click rate and the estimated conversion rate of each piece of displayable information corresponding to the roughing sorting model in the fourth time period, acquiring the estimated click number and the estimated conversion number of each piece of displayable information in the fourth time period, wherein the estimated click rate and the estimated conversion rate correspond to the roughing sorting model;

Based on the estimated number of clicks of each displayable information in the fourth time period corresponding to the carefully chosen sorting model and the estimated number of clicks of each displayable information in the fourth time period corresponding to the rough sorting model, acquiring click rate calibration coefficient target values of each displayable information corresponding to the rough sorting model respectively;

And based on the estimated conversion number of each displayable information in the fourth time period corresponding to the carefully chosen sorting model and the estimated conversion number of each displayable information in the fourth time period corresponding to the rough sorting model, acquiring conversion rate calibration coefficient target values of each displayable information corresponding to the rough sorting model respectively.

Optionally, the fourth obtaining unit is configured to perform weighted summation on the initial value of the second calibration coefficient and the calibration coefficient target value of the rough sorting model corresponding to each piece of displayable information, so as to obtain the calibration coefficient of the rough sorting model corresponding to each piece of displayable information.

In another aspect, a computer device is provided, the computer device comprising a processor and a memory having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, the at least one instruction, the at least one program, the set of codes, or the set of instructions being loaded and executed by the processor to implement the information presentation method as described above.

In another aspect, a computer readable storage medium having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions loaded and executed by a processor to implement an information presentation method as described above is provided.

The technical scheme provided by the application can comprise the following beneficial effects:

In the information display process, the calibration coefficients of each sub-ranking model can be obtained through the actual display results of each displayable information and the prediction display results of each cascaded sub-ranking model of the displayable information ranking model so as to calibrate the displayable information ranking model, and the information can be displayed based on the calibrated ranking results of the displayable information ranking model, so that the update of the displayable information ranking model based on the prediction effect and the actual display effect of the cascaded multi-stage displayable information ranking model is realized, and the accuracy of the displayable information ranking model is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application as claimed.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.

FIG. 1 illustrates a workflow diagram of an advertising trading platform of a multi-level ranking model, as shown in an exemplary embodiment of the application;

FIG. 2 illustrates a schematic diagram of the relationship among an advertising exchange, a media host, and an advertiser, according to an exemplary embodiment of the present application;

FIG. 3 illustrates a flowchart of training a multi-level ranking model in accordance with an exemplary embodiment of the present application;

FIG. 4 is a schematic diagram of an information presentation system according to an exemplary embodiment;

FIG. 5 illustrates a flow chart of a method for information presentation provided by an exemplary embodiment of the present application;

FIG. 6 is a flowchart of an information presentation method according to an exemplary embodiment of the present application;

FIG. 7 illustrates a schematic diagram of a real-time data flow multi-stage calibration algorithm framework, as illustrated in an exemplary embodiment of the present application;

Fig. 8 shows a block diagram of an information presentation apparatus according to an exemplary embodiment of the present application;

fig. 9 is a block diagram of a computer device, according to an example embodiment.

Detailed Description

Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the application. Rather, they are merely examples of apparatus and methods consistent with aspects of the application as detailed in the accompanying claims.

The embodiment of the application provides an information display method, which can calibrate a displayable information sequencing model through the actual display effect of displayable information on the basis of the displayable information sequencing model trained by artificial intelligence (ARTIFICIAL INTELLIGENCE, AI), so that the accuracy of the model is improved. For ease of understanding, the terms involved in the present application are explained below.

1) Artificial intelligence

Artificial intelligence is the theory, method, technique and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend and expand human intelligence, sense the environment, acquire knowledge and use the knowledge to obtain optimal results. In other words, artificial intelligence is an integrated technology of computer science that attempts to understand the essence of intelligence and to produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence, i.e. research on design principles and implementation methods of various intelligent machines, enables the machines to have functions of sensing, reasoning and decision.

The artificial intelligence technology is a comprehensive subject, and relates to the technology with wide fields, namely the technology with a hardware level and the technology with a software level. Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.

2) Machine learning (MACHINE LEARNING, ML)

Machine learning is a multi-domain interdisciplinary, involving multiple disciplines such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, and the like. It is specially studied how a computer simulates or implements learning behavior of a human to acquire new knowledge or skills, and reorganizes existing knowledge structures to continuously improve own performance. Machine learning is the core of artificial intelligence, a fundamental approach to letting computers have intelligence, which is applied throughout various areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, induction learning, teaching learning, and the like.

3) Media main

Media owners refer to entities that have an internet platform, which may be a platform or a person, such as WeChat friends circle, public numbers, news applications, electronic newspapers and magazines, etc., that typically have a large user access (also referred to as user traffic). For media owners, the user access may be converted into cash revenue by inserting ad slots in the platform.

4) Advertiser

An advertiser refers to an entity that presents its ad through an ad slot of an internet platform. For example, brands of advertisements are displayed in WeChat friends circles, or entities of advertisements are printed on advertisement boards of electronic newspapers and magazines, and the advertisements are displayed in advertisement places of a networking platform so as to transfer information related to the advertisements, thereby achieving the purpose of attracting users.

5) Online advertisement

Online advertisements, also referred to as internet advertisements, refer to advertisements placed on an advertisement spot (e.g., weChat circle, public number, news application, electronic journal, etc.) of an internet platform, which may include picture advertisements, text advertisements, keyword advertisements, rank advertisements, video advertisements, etc.

6) Advertisement transaction platform (ADX, ad Exchange)

An ad trading platform refers to an entity that links media owners and advertisers. It delivers the advertiser's advertisement to the ad slot provided by the media owner. In order to accurately deliver an advertiser's advertisement to a target crowd, the advertisement transaction platform typically collects user information for user portrayal, thereby accurately delivering advertisements with respect to the interests, geographic locations, or other data of the user.

7) Click through rate (CTR, click Through Rate)

The Click through rate refers to the Click arrival rate of an online advertisement, i.e., the actual number of clicks (Click) of the advertisement divided by the advertisement's presentation amount (Show content), which is an important indicator for measuring the effectiveness of the online advertisement.

The calculation formula can be expressed as:

8) Lightweight estimated click rate (LiteCTR, lite Predict Click TthroughRate)

The light estimated click rate is the probability that the online advertisement system estimates the online advertisement to be clicked after the online advertisement is put in a certain situation, is an important component of the sorting model, and represents the estimated click rate model in rough sorting under a multi-stage sorting model, and the complexity of the model is low.

9) Estimated click Rate (pCTR, predict Click Through Rate)

The estimated click rate is the probability that the online advertisement system estimates the online advertisement to be clicked after the online advertisement is put under a certain condition, is an important component of the sorting model, and represents the estimated click rate model in the carefully sorted sorting under the multi-stage sorting model, and the complexity of the model is high.

10 Conversion (CVR, conversion Rate)

Conversion rate is an indicator of the effectiveness of an online advertisement, and refers to the conversion rate of a user from clicking on the online advertisement to becoming an active, registered or paid user, i.e., the actual conversion number of the online advertisement divided by the click-through number of the online advertisement.

11 Light weight estimated Conversion (LiteCVR, lite Predict Conversion Rate)

The lightweight estimated conversion rate is the probability that an online advertisement system estimates conversion after being clicked under a certain condition, is an important component of a sorting model, and represents a conversion rate model in rough sorting under a multi-level sorting model, and the complexity of the model is low.

12 Estimated conversion (pCVR, predict Conversion Rate)

The estimated conversion rate is the probability that the online advertisement system estimates the conversion of the online advertisement after the online advertisement is clicked under a certain condition, is an important component of a sorting model, and represents the conversion rate model in carefully chosen sorting under a multi-stage sorting model, so that the complexity of the model is high.

13 Thousands of costs (CPM, cost Per Mille)

Thousands of people cost means cost that an advertiser needs to pay after online advertisements are displayed to thousands of access users on an internet platform, and a calculation formula is as follows:

Thousand people cost= (advertising cost/number of people reached) x 1000;

Wherein, the advertisement expense/number of arrival people is usually expressed in a percentage, the number of arrival people refers to the number of users who effectively access the online advertisement, for example, the cost paid by an advertiser for a certain online advertisement is 10000 yuan, the number of arrival people is 5000000, and then the thousands of people cost is:

10000/5000000 x 1000=2 (elements);

14 Bid (bid)

Bid refers to the price that the advertiser bid for an online advertisement, typically a converted price in oCPM (Optimized Cost Per Mille, optimizing thousand costs).

15 OCPM (Optimized Cost Per Mille, optimized thousand people cost)

OCPM the same charging mode as CPM is that the online advertising platform judges the value of each advertisement by the user. In this mode, the ad group sets the conversion target and cost price of the ad, and the online advertising platform optimizes the advertising according to the advertiser's settings, achieving the target as efficiently as possible. The charge after each thousand times of display of the online advertisement is positively correlated with the real-time bid of the online advertisement, wherein the real-time thousand people cost of the advertisement is as follows:

real-time CPM = bid x pCVR x pCTR;

16 Multi-level ranking model

Because of the large number of online advertisements, multiple (e.g., 2, including roughing and refining) pctrs of varying complexity are typically implemented based on engineering efficiency considerations, with the pCVR model calculating CPM for step-by-step screening of online advertisements that best meet the interests of the user, advertiser, and media. Wherein:

rougher-ordered real-time CPM = bid LiteCTR x LiteCVR;

carefully select rank-order real-time CPM = bid x pCTR x pCVR;

when the online advertisement is put on the internet of things platform, the online advertisement is usually put on a bidding mode, and the bidding advertisement has a plurality of bidding modes, and an advertiser can choose to bid according to exposure (CPM, costPer Mille), bid according to clicking (CPC, cost Per Click) or bid according to conversion (CPA, cost Per Action). Different bidding modes have different applicable scenes, for example, the main stream bidding mode of searching advertisement is CPC. Whereas mobile application class (APP) advertising is more concerned about conversion costs, the CPA approach is advantageous for the promotional information provider to control conversion costs.

With the development of advertisement delivery, bidding modes such as oCPM (Optimized Cost Per Mille), oCPA (Optimized Cost Per Action) and the like are gradually derived according to the bidding modes of conversion bidding besides CPA. Invariably CPA, oCPM, oCPA is bid on conversion. Obviously, in the process of advertisement delivery according to conversion bid, the advertisement trading platform needs to know the actual conversion quantity of the advertisement, so that the advertisement trading platform not only can be used for deduction and balance control, but also can optimize the advertisement effect in real time according to the actual conversion quantity, thereby delivering the advertisement to more suitable people.

The current advertisement trading platform sorts and screens advertisements put on advertisement slots of the internet platform through a multi-stage sorting model. Referring to fig. 1, which is a flowchart illustrating an advertisement trading platform of a multi-level ranking model according to an exemplary embodiment of the present application, as shown in fig. 1, the workflow of the advertisement trading platform includes:

step 110, a user request is received.

The user request refers to a request of a user to access an internet platform of a media owner and a request of the media owner to display an online advertisement, which are received by an advertisement transaction platform, and refer to fig. 2, which shows a schematic diagram of a relationship among the advertisement transaction platform, the media owner and the advertiser, which is shown in an exemplary embodiment of the present application, and as shown in fig. 2, the advertisement transaction platform 220 puts an advertisement of the advertiser 210 on an advertisement space of the media owner 230, and at the same time, can collect user information corresponding to the media owner, perform user portraits, and perform targeted casting on advertisements of the advertiser according to corresponding different user images of different media owners.

Step 120, advertisement rougher ordering.

When the advertisement transaction platform obtains a user request, the LiteCTR, liteCVR value of each advertisement in the roughing ordering model is calculated by combining the user information and the advertisement information, and the roughing real-time bidding, namely the roughing real-time CPM, of each advertisement is calculated according to the LiteCTR, liteCVR value. And the advertisement transaction platform sorts the advertisements according to the final rough concentration real-time CPM, and selects N advertisements with the highest sorting order to feed back to the carefully-selected sorting model, wherein N is a positive integer.

Step 130, advertisement selection ordering.

When the advertisement transaction platform obtains a user request, the pCTR and pCVR values of each advertisement in the carefully chosen sequencing model are calculated by combining the user information and the advertisement information, and the real-time bid of each advertisement is calculated according to the pCTR and pCVR values. And the advertisement trading platform sorts the N advertisements fed back by the coarse sorting model according to the final selected real-time bidding, namely the selected real-time CPM, and selects M advertisements with the highest sorting to send to the media host, wherein M is a positive integer.

And 140, winning advertisement display.

The media host receives the advertisement information provided by the advertisement trading platform, namely M advertisements screened by the advertisement trading platform through the rough sorting model and the carefully sorting model, and displays the advertisements on the advertisement positions of the media host. When the displayed advertisements accumulate enough display times in the media host, the advertisement transaction platform can charge the corresponding fees to the advertisement host according to the display times.

And 150, clicking and converting the data to reflux.

The advertisement transaction platform collects historical placement records and click and conversion records of placed advertisements. A check of the effect of the delivery system is made and the next iteration of the new model.

In the prior art, the above advertisement roughing ranking and advertisement selecting ranking may be completed by forming a multi-stage ranking model by a roughing ranking model and a selecting ranking model, the multi-stage ranking model may be applied in a server of an advertisement trading platform, and a basic flow for estimating CTR and CVR is implemented by using a machine learning algorithm, please refer to fig. 3, which illustrates a flowchart for training the multi-stage ranking model according to an exemplary embodiment of the present application, as shown in fig. 3, the flow may include the following steps:

and step 310, data processing.

Optionally, before performing data processing, the advertisement transaction platform needs to collect user information and acquire advertisement information, where the user information may be behavior information of the user on each media host platform and other internet platforms, personal attribute information of the user, smart device information of the user, clicking information of the user, converting advertisement, and so on.

And denoising and filling missing values of the collected user information. The image processing algorithm, natural language processing algorithm, machine learning algorithm, etc. are used to extract the interest features of the corresponding user from the user information, semantic features from the text, image, etc. of the advertisement. And generating a binary group < X, y > for the behavior record of the user on the advertisement by combining the interest feature of the user and the semantic feature of the advertisement, wherein X= (X ₁,x₂,…,x_m) comprises m features of the user and the advertisement, such as the attribute feature of the user, the behavior feature, the user interest preference feature extracted from the behavior, the attribute feature of the advertisement, the image feature, the character feature and the like, and y epsilon {1,0} is used for indicating whether the user clicks the advertisement or not in the click rate estimation task, that is, y=1 when the user clicks the advertisement, and y=0 when the user does not click the advertisement or closes the advertisement. In the conversion rate estimation task, y indicates whether the user converts the advertisement, where conversion refers to that the user clicks the advertisement to be a user who effectively activates, registers or pays for the corresponding strength of the advertisement, when the user converts the advertisement, y=1, and when the user does not convert the advertisement, that is, even if a certain user clicks the advertisement, the user does not perform activation, registration or payment, the user is still determined as not converting the advertisement, and y=0.

Step 320, model training.

By processing the user's behavioral records on the advertisement, the advertising trading platform generates a huge number of tuples < X _i,y_i >, i=1,..n, using a machine learning model, such as logistic regression, random forests, gradient-lifting trees, deep neural networks, and their variant algorithms, find an objective function f (X) such that y=f (X). Since in reality we do not know the specific form of f (X), machine learning algorithms will generally find an optimal f (X) by solving the following optimization equation:

Where L (·) is a loss function, used to measure the difference between y and f (X), a general loss function can be defined as a logarithmic loss function or a cross entropy loss, etc. k is the number of advertisements, n _i is the number of samples of the doublet of the ith advertisement, and the function which minimizes the solution of the optimization equation is selected as the objective function, namely the multi-level sequencing model is obtained.

And 330, evaluating the model.

Also called online evaluation, the quality of the multi-level sequencing model obtained in the training stage is evaluated, including running performance, estimated accuracy and the like.

And step 340, calibrating the model.

The multi-level ranking model obtained during the training phase after the evaluation is calibrated using a validation set of model evaluations, wherein the validation set is an offline data set.

Step 350, online deployment.

And further deploying the multi-stage sequencing model to an advertisement transaction platform to estimate click rate and conversion rate of the combination of the user and the advertisement through quality evaluation and calibration.

Because the verification set of model evaluation is used for calibrating the model in the process of training the multistage sequencing model, the method is only suitable for calibrating the single-stage sequencing model through the pre-acquired verification data set in the training process, is not suitable for the multistage sequencing model, and can not change the model in the subsequent model application process, so that the accuracy of the sequencing model is poor.

Fig. 4 is a schematic diagram illustrating a structure of an information presentation system according to an exemplary embodiment. The system includes an information publisher terminal 420, an information presentation device 440, and a server 460.

The information publisher terminal 420 may be a terminal device with network access capabilities and user interface display and interaction functions. For example, the information publisher terminal 420 may be a PC (such as a laptop or desktop computer, etc.), a smart phone, a tablet computer, an electronic book reader, etc.

The information presentation device 440 may be a computer device that includes or is externally connected to an information presentation platform, for example, the information presentation device 440 may be a cell phone, a tablet computer, an electronic book reader, smart glasses, a smart watch, an MP3 player (Moving Picture Experts Group Audio Layer III, dynamic video expert compression standard audio plane 3), an MP4 (Moving Picture Experts Group Audio Layer IV, dynamic video expert compression standard audio plane 4) player, a laptop portable computer, a desktop computer, a television set-top box, a game console, an outdoor advertisement presentation screen, a car advertisement presentation screen, and the like.

The information publisher terminal 420 and the information presentation apparatus 440 are connected to the server 460 via communication networks, respectively. Optionally, the communication network is a wired network or a wireless network.

Server 460 is a server, or is made up of several servers, or is a virtualization platform, or is a cloud computing service center.

The server 460 may include a resource vending platform 460a, an order management platform 460b, and an information management platform 460c, among others.

The resource selling platform 460a is configured to interact with the information publisher terminal 420, provide a service for querying the subscribed information display resource to the information publisher terminal 420 according to a user operation, and provide a service for locking/subscribing the information display resource to the information publisher terminal 420.

The order management platform 460b is configured to store and maintain orders for information publishers to order information display resources, such as creating orders for information publishers to request to order information display resources, deleting orders that have completed information display tasks (such as reaching exposure times) from existing orders, and so on.

The information management platform 460c is configured to manage the presentation of information issued by an information issuer, for example, push information issued by the information issuer to a corresponding information presentation platform according to a type of information presentation resource subscribed by the information issuer, and count exposure conditions of the information in each information presentation platform, and so on.

Optionally, the system may further include a management device (not shown in fig. 4) connected to the server 460 through a communication network. Optionally, the communication network is a wired network or a wireless network.

Alternatively, the wireless network or wired network described above uses standard communication techniques and/or protocols. The network is typically the Internet, but may be any network including, but not limited to, a local area network (Local Area Network, LAN), metropolitan area network (Metropolitan Area Network, MAN), wide area network (Wide Area Network, WAN), mobile, wired or wireless network, private network, or any combination of virtual private networks. In some embodiments, data exchanged over the network is represented using techniques and/or formats including HyperText Mark-up Language (HTML), extensible markup Language (Extensible Markup Language, XML), and the like. All or some of the links may also be encrypted using conventional encryption techniques such as secure sockets layer (Secure Socket Layer, SSL), transport layer security (Transport Layer Security, TLS), virtual private network (Virtual Private Network, VPN), internet protocol security (Internet Protocol Security, IPsec), etc. In other embodiments, custom and/or dedicated data communication techniques may also be used in place of or in addition to the data communication techniques described above.

Referring to fig. 5, which is a flowchart illustrating an information presentation method according to an exemplary embodiment of the present application, the information presentation method may be performed by a server, which may be the server shown in fig. 4, and the method may include the steps of:

Step 510, obtaining an actual display result of each piece of displayable information, wherein the displayable information is obtained by predicting the prediction probability of receiving the operation of the specified user through a displayable information sorting model and selecting the information to be displayed according to the prediction probability obtained by prediction, the displayable information sorting model comprises at least two cascaded sub-sorting models, and the actual display result refers to the actual data of the operation of the specified user received after each piece of displayable information is displayed in a display platform.

The presentable information is information that the terminal actively recommends to the user, wherein the presentable information can be represented as information such as articles, advertisements, music and the like.

The thousand-person cost is used to represent the cost that the corresponding advertiser needs to pay after displaying the advertisement to one thousand users accessing the internet platform on the internet platform. When an advertisement is put on the internet of things platform, the bidding mode is generally adopted for putting, the higher the thousand-person cost corresponding to the advertisement is, the higher the exposure rate of the advertisement in the internet of things platform is, and the thousand-person cost is positively correlated with the bidding price, the predicted click rate and the predicted conversion rate, so that the accuracy of the prediction of the predicted click rate and the predicted conversion rate influences the calculation result of the thousand-person cost, and further influences the actual display result of the displayable information.

The actual display result of each piece of displayable information refers to actual data of the appointed operation of the display platform user, which can be received by each piece of displayable information, in the process of actually displaying each piece of display information by the display platform. The actual data may be a statistical result obtained by the display platform by collecting data information of each presentable information in a specified time period and counting the data information.

Alternatively, the presentable information ordering model may be a multi-level ordering model, wherein the training and workflow of the multi-level ordering model may refer to related content in the embodiments shown in fig. 1 and 3, and are not described herein.

The displayable information sorting model comprises at least two sub-sorting models which are cascaded, wherein the cascade refers to that hierarchical connection exists among the sub-sorting models, namely, the last sub-sorting model can influence the next sub-sorting model, and the calibration target and the output result of the next sub-sorting model can be understood to be changed due to the fact that the calibration target and the output result of the last sub-sorting model are changed.

Step 520, obtaining a prediction display result of each displayable information corresponding to at least two sub-ranking models, where the prediction display result is used to indicate a prediction probability that at least two sub-ranking models receive a specified user operation for each displayable information prediction.

The predicted display results of at least two sub-ranking models corresponding to each displayable information respectively refer to that after each displayable information is respectively input into the displayable information ranking models, each sub-ranking model in the displayable information ranking models predicts the click rate and conversion rate of the displayable information, predicts the ranking results of each displayable information based on the predicted click rate and conversion rate in combination with the bid price of the corresponding each displayable information, and each sub-ranking model can set a plurality of displayable information with the ranking results in front to be displayed in the corresponding internet of things platform, while other displayable information is not displayed.

Step 530, based on the actual display result of each displayable information and the predicted display result of each displayable information corresponding to at least two sub-ordering models, obtaining the calibration coefficients of each displayable information corresponding to at least two sub-ordering models.

In the embodiment of the application, the calibration refers to taking the calibration target of each sub-ranking model as the output target of each sub-ranking model, and adjusting the parameters in each sub-ranking model, namely generating the calibration coefficient, so that the predicted result of each sub-ranking model on the click rate and/or the conversion rate is as close as possible to the calibration target corresponding to each sub-ranking model.

Because cascade connection exists among all the sub-sequence models, and the real-time performance is realized, the calibration result corresponding to each sub-sequence model is generated according to the current real-time calibration target, and unified calibration of the sub-sequence models can be realized.

Step 540, based on the calibration coefficients of the at least two sub-ranking models, correcting the predicted probabilities of the at least two sub-ranking models for each displayable information.

Optionally, in a subsequent preset time period, according to the calibration coefficients of the at least two sub-ranking models corresponding to each displayable information, correcting the click rate and/or conversion rate output by each sub-ranking model, obtaining the prediction probability of each corrected displayable information, calculating the thousand cost of each displayable information based on the corrected prediction probability, and sequencing the cost to display each displayable information, so that the display effect of each displayable information is improved.

In summary, in the information display method provided by the embodiment of the application, in the information display process, the calibration coefficients of each sub-ranking model of the actual display result of each displayable information and the prediction display result of each cascaded sub-ranking model of the displayable information ranking model are used to calibrate the displayable information ranking model, and display each displayable information based on the calibrated ranking result of the displayable information ranking model, so that the update of the displayable information ranking model based on the prediction effect and the actual display effect of the cascaded multi-stage displayable information ranking model is realized, thereby improving the accuracy of the displayable information ranking model and further improving the display effect of each displayable information.

Referring to fig. 6, which is a flowchart illustrating an information presentation method according to an exemplary embodiment of the present application, the information presentation method may be performed by a server, which may be the server shown in fig. 4, and the method may include the steps of:

Step 610, obtaining an actual display result of each piece of displayable information, wherein the displayable information is obtained by predicting the prediction probability of receiving the operation of the specified user through a displayable information sorting model and selecting the information to be displayed according to the prediction probability obtained by prediction, the displayable information sorting model comprises at least two cascaded sub-sorting models, and the actual display result refers to the actual data of the operation of the specified user received after each piece of displayable information is displayed in a display platform.

Optionally, the at least two sub-ranking models include a rough ranking model, and a fine ranking model concatenated after the rough ranking model.

In the embodiment of the application, the information display method provided by the application is described by the information display method with the displayable information sorting model which is composed of a rough sorting model and a carefully selected sorting model.

Alternatively, the actual display result may include the actual exposure number, the actual click number, and the actual conversion number.

Step 620, obtaining a prediction display result of each displayable information corresponding to at least two sub-ranking models, where the prediction display result is used to indicate a prediction probability that at least two sub-ranking models receive a specified user operation for each displayable information prediction.

Optionally, the predicted display result may include a predicted click rate and a predicted conversion rate, where the predicted click rate and the predicted conversion rate may be a predicted click rate and a predicted conversion rate corresponding to each of the sub-ranking models respectively.

Step 630, based on the actual display result of each displayable information and the predicted display result of each displayable information corresponding to the selected sorting model, obtaining the calibration coefficient of each displayable information corresponding to the selected sorting model.

Optionally, the calibration coefficients include a click rate calibration coefficient and a conversion rate calibration coefficient.

Optionally, the process of obtaining the calibration coefficients of the selected ranking model corresponding to each presentable information may be implemented as follows:

Step 631, obtaining a first calibration coefficient initial value of the carefully chosen ranking model based on the actual display result of each displayable information and the predicted display result of each displayable information corresponding to the carefully chosen ranking model, respectively, and obtaining a calibration coefficient target value of each displayable information corresponding to the carefully chosen ranking model, respectively.

Since the refined ranking model acts on the rough ranking model, the target value of the calibration coefficient is the ranking result of the refined ranking model in the last time period for the current rough ranking model. Referring to fig. 7, a schematic diagram of a real-time data flow multi-level calibration algorithm framework according to an exemplary embodiment of the present application is shown, and as shown in fig. 7, after receiving a user request (S71), an advertisement transaction platform performs rough sorting on advertisements by combining user information and advertisement information (S72), selects the first N advertisements in which rough real-time CPM is the forefront, performs fine sorting on the first N advertisements (S73), and selects the first M advertisements in which fine real-time CPM is the forefront, and sends the first M advertisements to a media host. After the advertisement pull (S74) is performed by the media host, the advertisement is exposed (S75), and operations of clicking (S76) and converting (S77) are performed in response to the user' S advertisement.

The rough sorting is completed by the rough sorting model, the carefully sorting is completed by the carefully sorting model, when the carefully sorting model is calibrated, the calibration coefficient target value is calculated (S78) in real time according to at least one of the real-time exposure rate, the click rate and the conversion rate of advertisements, when the rough sorting model is calibrated, the sorting result of the carefully sorting model in the previous period is calibrated as the calibration coefficient target value, and as the calibration coefficient target value of the rough sorting model is the estimated result of the carefully sorting model, the estimated results of the rough sorting model and the carefully sorting model finally tend to be consistent, namely the sorting result of each advertisement by the rough sorting model and the sorting result of the advertisements fed back by the carefully sorting model tend to be consistent, so that the aim of joint calibration is achieved.

Optionally, the first calibration coefficient initial value of the selected ranking model may be obtained based on the actual display result of each of the displayable information in the first time period and the predicted display result of each of the displayable information in the first time period corresponding to the selected ranking model, respectively.

The process of obtaining the first calibration coefficient initial value of the carefully chosen sorting model is a process of initializing the prediction display result of the information-capable sorting model, and since the prediction display result corresponding to each information-capable sorting model is a value which changes with time or the change of the content of the media main platform, the prediction display result corresponding to each information-capable sorting model needs to be initialized in order to realize the calibration algorithm of the carefully chosen sorting model.

Alternatively, the process of obtaining the initial value of the first calibration coefficient may be implemented as follows:

S6311, obtaining the sum of actual click numbers of each displayable information in the first time period and the sum of actual conversion numbers of each displayable information in the first time period.

Alternatively, the first period may be K hours nearest to the point in time at which the sum of the click numbers is acquired, K being a positive number.

S6312, obtaining a sum of first estimated click numbers of the displayable information in the first time period and a sum of first estimated conversion numbers of the displayable information in the first time period, which correspond to the carefully-selected sorting models, based on actual exposure numbers of the displayable information in the first time period and the estimated click rates and the estimated conversion rates of the displayable information in the first time period, respectively.

The estimated click rate refers to the probability that the displayable information ordering model estimates the click after a certain displayable information is put in a certain situation, and the estimated conversion rate refers to the probability that the displayable information ordering model estimates the conversion after a certain displayable information is put in a certain situation.

Optionally, a first sampling period in the first period may be preset, and the estimated click rate and the estimated conversion number of each displayable information in the first period may be obtained according to the first sampling period, so as to calculate the total estimated click number and the total estimated conversion number of each displayable information in the first period.

For example, if the preset sampling period in K hours is 5 minutes, the estimated click rate and the estimated conversion rate of each message are obtained once every 5 minutes, and the estimated click number and the estimated conversion number in 5 minutes are calculated, so that the total estimated click number and the total estimated conversion number in K hours are calculated.

It should be noted that the sampling period may be set according to actual requirements, which is not limited by the present application.

S6313, obtaining a first click rate calibration coefficient initial value of the selected sorting model based on the sum of actual click times of each displayable information in the first time period and the sum of first estimated click times.

In the calculation process of the estimated click rate and the click rate, the estimated click rate is obtained by calculating the estimated click number divided by the actual exposure number, the click rate is obtained by calculating the actual click number divided by the actual exposure number, therefore, when the first click rate calibration coefficient initial value is calculated, the actual exposure number can be divided by the total actual click rate divided by the total estimated click rate, therefore, the calculated first click rate calibration coefficient initial value can be expressed as the sum of the actual click number in the first time period divided by the sum of the estimated click number in the first time period, and the calculation formula can be expressed as follows:

Wherein fix_ pctr _ratio (0) represents the initial value of the first click rate calibration coefficient, Σclick _i (0) represents the sum of the actual number of clicks in the first time period, Σ pctr _i (0) represents the sum of the estimated number of clicks in the first time period, and i represents the ith presentable information.

S6314, obtaining a first conversion rate calibration coefficient initial value of the carefully chosen sorting model based on the sum of actual conversion numbers of each displayable information in the first time period and the sum of first estimated conversion numbers.

In the calculation process of the estimated conversion rate and the conversion rate, the estimated conversion rate is obtained by dividing the estimated conversion rate by the actual click rate, the conversion rate is obtained by dividing the actual conversion rate by the actual click rate, so that when the initial value of the first conversion rate calibration coefficient is calculated, the actual click rate can be reduced when the total actual conversion rate is divided by the total estimated conversion rate, and therefore, the calculated initial value of the first conversion rate calibration coefficient can be expressed as the sum of the actual conversion rate in the first time period divided by the sum of the estimated conversion rate in the first time period, and the calculation formula can be expressed as follows:

wherein fix_ pcvr _ratio (0) represents the initial value of the first conversion rate calibration coefficient, Σconv _i (0) represents the sum of the actual conversion numbers in the first time period, Σ pcvr _i (0) represents the sum of the estimated conversion numbers in the first time period, and i represents the ith presentable information.

Optionally, the calibration coefficient target value of each displayable information corresponding to the carefully chosen ranking model may be obtained based on the actual display result of each displayable information in the second time period.

Alternatively, the second time period may be on the order of minutes to ensure the instantaneity of the pick ranking model calibration algorithm, such as the first time period may be the last 1 minute.

Optionally, the target value of the calibration coefficient of the carefully selected ranking model may include at least one of an exposure rate, a click rate, and a conversion rate of the presentable information;

The exposure rate of the displayable information refers to the probability that the displayable information is displayed to a media host user, the click rate refers to the proportion of users clicking the displayable information among users who receive pushing of certain popularization information, the conversion rate refers to the proportion of users who make specific behaviors on the displayable information among users clicking certain displayable information, the specific behaviors can be set according to actual conditions, for example, if the displayable information is an article, the users can click to read the article as corresponding specific behaviors, the users click to read the article is marked as converted once, if the displayable information is an advertisement of a mobile application APP, the specific behaviors can be set as behaviors for users to download the APP, and the APP is marked as converted once.

For example, in a period of time, 1000 users of a media owner push a certain displayable information to one of the end users by the advertisement transaction platform, the displayable information is exposed once, the exposure rate of the displayable information is 10% assuming that 100 users receive the displayable information in the period of time, the click rate of the displayable information is 50% assuming that 50 end users in 100 end users browsing the displayable information click the displayable information, and the conversion rate of the displayable information is 20% assuming that 10 end users do specific actions on the displayable information in 50 end users clicking the displayable information.

In the embodiment of the application, the information display method provided by the application is described by taking the target value of the calibration coefficient of the carefully selected sorting model as the click rate and the conversion rate.

Optionally, the process of obtaining the calibration coefficient target value of each exposable message corresponding to the carefully chosen ranking model may be implemented as follows:

S6315, obtaining the actual click number of each displayable information in the second time period and the actual conversion number of each displayable information in the second time period.

Optionally, a second sampling period in the second time period may be preset, and the actual number of clicks and the actual number of conversions of each displayable information in each second sampling period in the second time period are obtained according to the second sampling period, so as to calculate the actual number of clicks of each displayable information in the second time period and the actual number of conversions in the second time period, where a calculation formula may be expressed as follows:

sum_click(t)=∑click_i(t)

sum_conv(t)=∑conv_i(t)

Where t represents the second time period, sum_click (t) represents the actual number of clicks in the second time period, click _i (t) represents the actual number of turns in each second sampling period.

S6316, based on the actual exposure number of each displayable information in the second time period and the estimated click rate and the estimated conversion rate of each displayable information corresponding to the carefully chosen sorting model in the second time period, obtaining the estimated click number and the estimated conversion number of each displayable information corresponding to the carefully chosen sorting model in the second time period.

Optionally, the estimated click rate and the estimated conversion rate of each displayable information in each second sampling period of the second time period are obtained according to the second sampling period, so that the estimated click number and the estimated conversion number in each second sampling period are calculated by combining the actual exposure number of each displayable information in each second sampling period of the second time period, and thus the estimated click number and the estimated conversion number of each displayable information in the second time period are calculated, and a calculation formula can be expressed as follows:

sum_pctr(t)=∑pctr_i(t)

sum_pcvr(t)=∑pcvr_i(t)

Where t represents a second time period, sum_pctr (t) represents a predicted number of clicks in the second time period, pctr _i (t) represents a predicted number of clicks in each second sampling period, sum_pcvr (t) represents a predicted number of transitions in the second time period, and pcvr _i (t) represents a predicted number of transitions in each second sampling period.

S6317, based on the actual click number of each displayable information in the second time period and the estimated click number of each displayable information in the second time period corresponding to the carefully chosen sorting model, obtaining a click rate calibration coefficient target value of each displayable information corresponding to the carefully chosen sorting model respectively, wherein the calculation formula can be expressed as follows:

Where t represents the second time period, update_ pctr _ratio _i (t) represents the click rate calibration coefficient target value of the pick ranking model.

S6318, based on the actual conversion number of each displayable information in the second time period and the estimated conversion number of each displayable information in the second time period corresponding to the carefully chosen sorting model, obtaining a conversion rate calibration coefficient target value of each displayable information corresponding to the carefully chosen sorting model respectively, wherein the calculation formula can be expressed as follows:

where t represents the second time period, update_ pcvr _ratio _i (t) represents the conversion calibration coefficient target value of the pick-order model.

Step 632, based on the first calibration coefficient initial value and the calibration coefficient target value of each displayable information corresponding to the carefully chosen ranking model, obtains the calibration coefficient of each displayable information corresponding to the carefully chosen ranking model.

Optionally, the first calibration coefficient initial value and the calibration coefficient target value of each displayable information corresponding to the carefully chosen sorting model are weighted and summed to obtain the calibration coefficient of each displayable information corresponding to the carefully chosen sorting model, and the calculation formula can be expressed as:

fix_pctr_ratio_i(t+1)＝(1-α)*fix_pctr_ratio_i(0)+α*update_pctr_ratio_i(t)

fix_pcvr_ratio_i(t+1)＝(1-α)*fix_pcvr_ratio_i(t)+α*update_pcvr_ratio_i(t)

Where t+1 represents the next time period after the second time period, fix_ pctr _ratio _i (t+1) and fix_ pcvr _ratio _i (t+1) represent the calibration coefficients of the pick-and-sort model, α represents the smoothing coefficients to balance the first click rate calibration coefficient initial value and the click rate calibration coefficient target value of the pick-and-sort model, and to balance the first conversion rate calibration coefficient initial value and the conversion rate calibration coefficient target value of the pick-and-sort model.

In general, the smoothing coefficient α represents the response speed of the exponential smoothing model to time series changes, and determines the ability of the prediction model to smooth random errors. The value of α may be changed according to the service requirement, where the value is related to the number of samples in the first time, and generally, the greater the number of samples, the greater the value that α may be set.

Step 640, obtaining calibration coefficients of the rough sorting models corresponding to the displayable information respectively based on the prediction display results of the fine sorting models corresponding to the displayable information respectively and the prediction display results of the rough sorting models corresponding to the displayable information respectively.

Optionally, the process of obtaining the calibration coefficients of the rough sorting model corresponding to each displayable information respectively may be implemented as follows:

In step 641, based on the predicted display result of each displayable information corresponding to the selected sorting model, and the predicted display result of each displayable information corresponding to the rough sorting model, the initial value of the second calibration coefficient of the rough sorting model is obtained, and the calibration coefficient target value of each displayable information corresponding to the initial sorting model is obtained.

Optionally, the first calibration coefficient value of the rough sorting model may be obtained based on the predicted display result of each displayable information corresponding to the carefully chosen sorting model in the third time period and the predicted display result of each displayable information corresponding to the rough sorting model in the third time period.

Alternatively, the third time period may be the same time period as the first time period.

Optionally, the process of obtaining the initial value of the second calibration coefficient of the rough sorting model may be implemented as follows:

S6411, based on the actual exposure number of each displayable information in the third time period and the estimated click rate and the estimated conversion rate of each displayable information corresponding to the carefully chosen sorting model in the third time period, obtaining the sum of the second estimated click numbers of each displayable information in the third time period and the sum of the second estimated conversion numbers of each displayable information in the third time period, wherein the second estimated click rates and the estimated conversion rates correspond to the carefully chosen sorting model.

S6412, based on the actual exposure number of each displayable information in the third time period and the estimated click rate and the estimated conversion rate of each displayable information corresponding to the roughing sorting model in the third time period, obtaining the sum of the third estimated click numbers of each displayable information in the third time period and the sum of the third estimated conversion numbers of each displayable information in the third time period, wherein the third estimated click rates and the third estimated conversion rates correspond to the roughing sorting model respectively.

Alternatively, the estimated click rate of the rough sorting model may be referred to as a light-weight estimated click rate, and the estimated conversion rate of the rough sorting model may be referred to as a light-weight estimated conversion rate.

Optionally, a third sampling period in a third time period may be preset, and the light-weight estimated click rate and the light-weight estimated conversion rate of each displayable information in the third time period may be obtained according to the third sampling period, so as to calculate a total light-weight estimated click number and a total light-weight estimated conversion number of each displayable information in the third time period, where when the third time period and the first time period are the same time period, the third sampling period should be kept consistent with the first sampling period.

S6413, obtaining a second click rate calibration coefficient initial value of the roughing sorting model based on the sum of the second estimated click times of each displayable information in the third time period and the sum of the third estimated click times of each displayable information in the third time period.

In the calculation process of the light estimated click rate and the estimated click rate, the light estimated click rate is obtained by dividing the light estimated click number (third estimated click number) by the actual exposure number, the estimated click rate is obtained by dividing the estimated click number by the actual exposure number, when the second click rate calibration coefficient initial value is calculated, the actual exposure number can be divided by the total estimated click rate divided by the total light estimated click rate, therefore, the calculated second click rate calibration coefficient initial value can be expressed as the sum of the estimated click number in the third time period divided by the sum of the light estimated click number (third estimated click number) in the third time period, and the calculation formula can be expressed as follows:

Wherein fix_ litectr _ratio (0) represents the initial value of the second click rate calibration coefficient, Σ pctr _i (0) represents the sum of the second estimated number of clicks in the third time period, Σ litectr _i (0) represents the sum of the third estimated number of clicks in the third time period, and i represents the ith presentable information.

S6414, obtaining a second conversion rate calibration coefficient initial value of the roughing sorting model based on the sum of the second estimated conversion numbers of each displayable information in the third time period and the sum of the third estimated conversion numbers of each displayable information in the third time period.

Since the light-weight estimated conversion rate is obtained by dividing the light-weight estimated conversion rate (third estimated conversion rate) by the actual click rate in the calculation process of the light-weight estimated conversion rate and the estimated conversion rate, the estimated conversion rate is obtained by dividing the estimated conversion rate by the actual click rate, and when the second conversion rate calibration coefficient initial value is calculated, the actual click rate can be divided by the total light-weight estimated conversion rate, and therefore, the calculation formula of the second conversion rate calibration coefficient initial value can be expressed as the sum of the second estimated conversion rate in the third time period divided by the sum of the third estimated conversion rate in the third time period, and can be expressed as follows:

Wherein fix_ litecvr _ratio (0) represents the initial value of the second conversion rate calibration coefficient, Σ pcvr _i (0) represents the sum of the second estimated conversion in the third time period and the third estimated conversion in the third time period, and i represents the ith presentable information.

Optionally, the server may obtain the calibration coefficient target value of each exposable information corresponding to the rough sorting model respectively based on the predicted exponentiation result of each exposable information corresponding to the carefully sorted sorting model respectively in the fourth time period.

Alternatively, the fourth time period may be the same time period as the second time period.

Optionally, the process of obtaining the calibration coefficient target values of the rough sorting model corresponding to the respective displayable information may be implemented as follows:

S6415, based on the actual exposure number of each displayable information in the fourth time period and the estimated click rate and the estimated conversion rate of each displayable information corresponding to the carefully chosen sorting model in the fourth time period, acquiring the estimated click number and the estimated conversion number of each displayable information in the fourth time period corresponding to the carefully chosen sorting model.

Optionally, a fourth sampling period in the fourth time period may be preset, and the estimated click rate and the estimated conversion rate of the carefully selected sorting model of each displayable information in each fourth sampling period in the fourth time period may be obtained according to the fourth sampling period, so as to combine the actual exposure number of each displayable information in each fourth sampling period in the fourth time period, thereby calculating the estimated click number of each displayable information in the fourth time period and the estimated conversion number in the fourth time period. When the fourth time period and the second time period are the same, the fourth sampling period should also be consistent with the second sampling period.

When the fourth time period and the second time period are the same time period, the calculation formula can be expressed as follows:

sum_pctr_i(t)＝∑pctr_i(t)

sum_pcvr_i(t)＝∑pcvr_i(t)

Where t represents the fourth time period, sum_ pctr _i (t) represents the estimated number of clicks of the carefully chosen ranking model in the fourth time period, pctr _i (t) represents the estimated number of transitions of the carefully chosen ranking model in each sampling period.

S6416, based on the actual exposure number of each displayable information in the fourth time period and the estimated click rate and the estimated conversion rate of each displayable information corresponding to the rough sorting model in the fourth time period, acquiring the estimated click number and the estimated conversion number of each displayable information in the fourth time period corresponding to the rough sorting model.

Optionally, the light-weight estimated click rate and the light-weight estimated conversion rate of each displayable information in each fourth sampling period of the fourth time period are obtained according to the fourth sampling period, so that the light-weight estimated click rate and the light-weight estimated conversion rate in each fourth sampling period are calculated by combining the actual exposure number of each displayable information in each fourth sampling period of the fourth time period, and thus the light-weight estimated click rate and the light-weight estimated conversion rate of each displayable information in the fourth time period are calculated, and the calculation formula can be expressed as follows:

sum_litectr_i(t)＝∑litectr_i(t)

sum_litecvr_i(t)＝∑litecvr_i(t)

Where t represents the fourth time period, sum_pctr (t) represents the number of light-weight estimated transitions in the fourth time period, pctr _i (t) represents the number of light-weight estimated transitions in each fourth sampling period, sum_pcvr (t) represents the number of light-weight estimated transitions in the fourth time period, and pcvr _i (t) represents the number of light-weight estimated transitions in each fourth sampling period.

S6417, based on the estimated click number of each displayable information in the fourth time period corresponding to the carefully chosen sorting model and the estimated click number of each displayable information in the fourth time period corresponding to the rough sorting model, acquiring a click rate calibration coefficient target value of each displayable information corresponding to the rough sorting model respectively, wherein a calculation formula can be expressed as follows:

Wherein t represents a fourth time period, and update_ litectr _ratio (t) represents a click rate calibration coefficient target value of the roughing sorting model.

S6418, based on the estimated conversion number of each displayable information in the fourth time period corresponding to the carefully chosen sorting model and the estimated conversion number of each displayable information in the fourth time period corresponding to the rough sorting model, obtaining a conversion rate calibration coefficient target value of each displayable information corresponding to the rough sorting model respectively, wherein the calculation formula can be expressed as follows:

Wherein t represents a fourth time period, and update_ litecvr _ratio (t) represents a conversion rate calibration coefficient target value of the roughing sorting model.

Step 642, based on the initial value of the second calibration coefficient and the calibration coefficient target value of the initial sorting model corresponding to each displayable information, obtaining the calibration coefficient of the rough sorting model corresponding to each displayable information.

Optionally, the first calibration coefficient value and the calibration coefficient target value of the roughing sorting model corresponding to each piece of displayable information are weighted and summed to obtain the calibration coefficient of the roughing sorting model corresponding to each piece of displayable information, and the calculation formula can be expressed as follows:

fix_litectr_ratio_i(t+1)＝(1-α)*fix_litectr_ratio_i(t)+α*update_litectr_ratio_i(t)

fix_litecvr_ratio_i(t+1)＝(1-α)*fix_litecvr_ratio_i(t)+α*update_litecvr_ratio_i(t)

Where t+1 represents the next time period after the fourth time period, fix_ pctr _ratio _i (t+1) and fix_ pcvr _ratio _i (t+1) represent the calibration coefficients of the roughing ranking model, α represents the smoothing coefficient to balance the initial value of the second click rate calibration coefficient with the target value of the click rate calibration coefficient of the roughing ranking model, and balance the initial value of the first conversion rate calibration coefficient with the target value of the conversion rate calibration coefficient of the roughing ranking model.

Step 650, correcting the prediction probability of each exposable message for at least two sub-ranking models based on the calibration coefficients of at least two sub-ranking models.

Optionally, for each selected sorting model capable of displaying the information sorting model, after calibration is performed by combining with the corresponding calibration coefficient, a formula for calculating the real-time thousand cost is expressed as follows:

carefully chosen rank-ordered real-time cpm=bid× pCVR ×pctr x_ pCTR _ratio _i(t+1)*fix_pcvr_ratio_i (t+1)

The pCVR ×pctr x_ pCTR _ratio _i(t+1)*fix_pcvr_ratio_i (t+1) represents the prediction probability of the modified selection ordering model on each displayable information, and the selection ordering real-time thousand-person cost of each displayable information is calculated based on the modified prediction probability, so that each displayable information is reordered, and the displaying mode of each displayable information is determined.

For each rough sorting model capable of showing the information sorting model, after the rough sorting model is calibrated by combining the corresponding calibration coefficients, the formula for calculating the real-time thousand-person cost is expressed as follows:

rougher-ordered real-time cpm=bid_ LiteCVR _ LiteCTR _fix_ litectr _ratio _i(t+1)*fix_litecvr_ratio_i (t+1)

The LiteCVR × LiteCTR ×fix_ litectr _ratio _i(t+1)*fix_litecvr_ratio_i (t+1) represents the prediction probability of the modified rough sorting model on each displayable information, and calculates the rough sorting real-time thousand-person cost of each displayable information based on the modified prediction probability, so as to reorder each displayable information and further determine the display mode of each displayable information.

Optionally, there is a difference between the calibration coefficients of the selected ranking model corresponding to the different displayable information, and there is a difference between the calibration coefficients of the rough ranking model corresponding to the different displayable information.

In the practical application process of the information sequence model, the calibration process of the carefully selected sequence model and the rough selected sequence model is performed in real time according to real-time data, so that the accuracy of the information sequence model can be displayed.

In summary, in the information display method provided by the embodiment of the application, in the information display process, the calibration coefficient of each sub-ranking model can be obtained through the actual display result of each displayable information and the prediction display result of each cascaded sub-ranking model of the displayable information ranking model, so as to calibrate the displayable information ranking model, display each displayable information based on the ranking result of the calibrated displayable information ranking model, and update the displayable information ranking model based on the prediction effect and the actual display effect of the cascaded multi-stage displayable information ranking model, thereby improving the accuracy of the displayable information ranking model and further improving the display effect of each displayable information.

Referring to fig. 8, which is a block diagram illustrating an information presentation apparatus according to an exemplary embodiment of the present application, the information presentation apparatus may be applied to a server, which may be implemented as the server shown in fig. 1, and the apparatus may include:

The first obtaining module 810 is configured to obtain an actual display result of each displayable information, where the displayable information is information that is displayed by predicting a prediction probability of receiving a specified user operation through a displayable information ordering model and selecting according to the prediction probability obtained by the prediction;

a second obtaining module 820, configured to obtain a prediction display result of each displayable information corresponding to at least two sub-ranking models, where the prediction display result is used to indicate a prediction probability that at least two sub-ranking models receive a specified user operation for each displayable information prediction;

A third obtaining module 830, configured to obtain calibration coefficients of at least two sub-ranking models corresponding to each displayable information based on an actual display result of each displayable information and a predicted display result of each displayable information corresponding to at least two sub-ranking models respectively;

The presentation module 840 is configured to correct the prediction probabilities of the at least two sub-ranking models for each presentable information based on the calibration coefficients of the at least two sub-ranking models.

Optionally, the at least two sub-ranking models include a rough ranking model, and a carefully selected ranking model cascaded after the rough ranking model;

the second acquisition module 820 includes:

the first acquisition sub-module is used for acquiring calibration coefficients of the selected sorting models corresponding to the information capable of being displayed respectively based on the actual display results of the information capable of being displayed and the predicted display results of the selected sorting models corresponding to the information capable of being displayed respectively;

The second obtaining sub-module is used for obtaining the calibration coefficients of the rough sorting model corresponding to the displayable information respectively based on the prediction display results of the fine sorting model corresponding to the displayable information respectively and the prediction display results of the rough sorting model corresponding to the displayable information respectively.

Optionally, the first obtaining sub-module includes:

The first acquisition unit is used for acquiring a first calibration coefficient initial value of the carefully chosen sequencing model and a calibration coefficient target value of the carefully chosen sequencing model respectively corresponding to each piece of displayable information based on the actual display result of each piece of displayable information and the predicted display result of each piece of displayable information respectively corresponding to the carefully chosen sequencing model;

the second obtaining unit is used for obtaining the calibration coefficient of each displayable information corresponding to the carefully chosen sequencing model respectively based on the first calibration coefficient initial value and the calibration coefficient target value of each displayable information corresponding to the carefully chosen sequencing model respectively.

Optionally, the first obtaining unit includes:

The first acquisition subunit is used for acquiring a first calibration coefficient initial value of the carefully chosen sorting model based on the actual display result of each piece of displayable information in the first time period and the prediction display result of each piece of displayable information corresponding to the carefully chosen sorting model in the first time period;

The first acquisition subunit is configured to:

Obtaining the sum of the actual click numbers of each displayable information in the first time period and the sum of the actual conversion numbers of each displayable information in the first time period;

Based on the actual exposure number of each displayable information in the first time period, and the estimated click rate and the estimated conversion rate of each displayable information corresponding to the carefully selected sorting model in the first time period, obtaining the sum of the first estimated click numbers of each displayable information in the first time period and the sum of the first estimated conversion numbers of each displayable information in the first time period, wherein the sum corresponds to the carefully selected sorting model;

Acquiring a first click rate calibration coefficient initial value of the carefully chosen sorting model based on the sum of the actual click numbers of each displayable information in the first time period and the sum of the first estimated click numbers;

and obtaining a first conversion rate calibration coefficient initial value of the carefully chosen sorting model based on the sum of the actual conversion numbers of each displayable information in the first time period and the sum of the first estimated conversion numbers.

the second acquisition subunit is configured to:

Acquiring the actual click number of each displayable information in a second time period and the actual conversion number of each displayable information in the second time period;

Based on the actual exposure number of each displayable information in the second time period and the estimated click rate and the estimated conversion rate of each displayable information corresponding to the carefully selected sorting model in the second time period, acquiring the estimated click number and the estimated conversion number of each displayable information in the second time period corresponding to the carefully selected sorting model;

and acquiring conversion rate calibration coefficient target values of the selected sorting models, which correspond to the information capable of being displayed, based on the actual conversion numbers of the information capable of being displayed in the second time period and the estimated conversion numbers of the information capable of being displayed in the second time period, which correspond to the selected sorting models.

Optionally, the second obtaining unit is configured to perform weighted summation on the initial value of the first calibration coefficient and the calibration coefficient target value of the selected sorting model corresponding to each piece of displayable information, so as to obtain the calibration coefficient of the selected sorting model corresponding to each piece of displayable information.

Optionally, the second obtaining sub-module includes:

The third acquisition unit is used for acquiring a second calibration coefficient initial value of the rough sorting model based on the prediction display result of each displayable information corresponding to the carefully chosen sorting model and the prediction display result of each displayable information corresponding to the rough sorting model, and the calibration coefficient target value of each displayable information corresponding to the initial sorting model;

the fourth obtaining unit is configured to obtain, based on the initial value of the second calibration coefficient and the calibration coefficient target value of the preliminary ranking model corresponding to each displayable information, the calibration coefficient of the rough ranking model corresponding to each displayable information.

Optionally, the third obtaining unit includes:

The third obtaining subunit is used for obtaining a second calibration coefficient initial value of the rough sorting model based on the prediction display result of each displayable information corresponding to the carefully chosen sorting model in the third time period and the prediction display result of each displayable information corresponding to the rough sorting model in the third time period;

the fourth obtaining subunit is configured to obtain, based on the predicted display results of the respective displayable information corresponding to the rough selection ranking model in the fourth time period, calibration coefficient target values of the respective displayable information corresponding to the rough selection ranking model.

the third obtaining subunit is configured to obtain, based on the actual exposure number of each displayable information in the third time period and the estimated click rate and the estimated conversion rate of each displayable information corresponding to the carefully-selected sorting model in the third time period, a sum of the second estimated click numbers of each displayable information in the third time period and a sum of the second estimated conversion numbers of each displayable information in the third time period, where the second estimated click rate and the estimated conversion rate correspond to the carefully-selected sorting model;

based on the actual exposure number of each displayable information in the third time period, and the estimated click rate and the estimated conversion rate of each displayable information corresponding to the rough sorting model in the third time period, respectively, obtaining the sum of the third estimated click numbers of each displayable information in the third time period and the sum of the third estimated conversion numbers of each displayable information in the third time period, wherein the sum corresponds to the rough sorting model;

Acquiring a second click rate calibration coefficient initial value of the rough sorting model based on the sum of the second estimated click numbers of each displayable information in the third time period and the sum of the third estimated click numbers of each displayable information in the third time period;

and obtaining a second conversion rate calibration coefficient initial value of the roughing sorting model based on the sum of the second estimated conversion numbers of each displayable information in the third time period and the sum of the third estimated conversion numbers of each displayable information in the third time period.

The fourth obtaining subunit is configured to obtain, based on the actual exposure number of each displayable information in the fourth time period and the estimated click rate and the estimated conversion rate of each displayable information corresponding to the carefully-selected sorting model in the fourth time period, the estimated click number and the estimated conversion number of each displayable information in the fourth time period corresponding to the carefully-selected sorting model;

Based on the actual exposure number of each displayable information in the fourth time period and the estimated click rate and the estimated conversion rate of each displayable information corresponding to the rough sorting model in the fourth time period, acquiring the estimated click number and the estimated conversion number of each displayable information in the fourth time period corresponding to the rough sorting model;

Based on the estimated click number of each displayable information in the fourth time period corresponding to the carefully chosen sorting model and the estimated click number of each displayable information in the fourth time period corresponding to the rough chosen sorting model, acquiring a click rate calibration coefficient target value of each displayable information corresponding to the rough chosen sorting model respectively;

Based on the estimated conversion number of each displayable information in the fourth time period corresponding to the carefully chosen sorting model and the estimated conversion number of each displayable information in the fourth time period corresponding to the rough sorting model, obtaining the conversion rate calibration coefficient target value of each displayable information corresponding to the rough sorting model respectively.

In summary, in the information display device provided by the embodiment of the application, in the information display process, the calibration coefficient of each sub-ranking model can be obtained through the actual display result of each displayable information and the prediction display result of each cascaded sub-ranking model of the displayable information ranking model, so as to calibrate the displayable information ranking model, display each displayable information based on the ranking result of the calibrated displayable information ranking model, and update the displayable information ranking model based on the prediction effect and the actual display effect of the cascaded multi-stage displayable information ranking model, thereby improving the accuracy of the displayable information ranking model and further improving the display effect of each displayable information.

Fig. 9 is a block diagram of a computer device 900, shown in accordance with an exemplary embodiment. The computer device may be implemented as a server in the above-described aspects of the present application. The computer apparatus 900 includes a central processing unit (Central Processing Unit, CPU) 901, a system Memory 904 including a random access Memory (Random Access Memory, RAM) 902 and a Read-Only Memory (ROM) 903, and a system bus 905 connecting the system Memory 904 and the central processing unit 901. The computer device 900 also includes a basic Input/Output system (I/O) 906, which helps to transfer information between various devices within the computer, and a mass storage device 907, for storing an operating system 913, application programs 914, and other program modules 915.

The basic input/output system 906 includes a display 908 for displaying information and an input device 909, such as a mouse, keyboard, etc., for user input of information. Wherein the display 908 and the input device 909 are connected to the central processing unit 901 via an input output controller 910 connected to the system bus 905. The basic input/output system 906 can also include an input/output controller 910 for receiving and processing input from a number of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, the input-output controller 910 also provides output to a display screen, a printer, or other type of output device.

The mass storage device 907 is connected to the central processing unit 901 through a mass storage controller (not shown) connected to the system bus 905. The mass storage device 907 and its associated computer-readable media provide non-volatile storage for the computer device 900. That is, the mass storage device 907 may include a computer readable medium (not shown) such as a hard disk or a compact disk-Only (CD-ROM) drive.

The computer readable medium may include computer storage media and communication media without loss of generality. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, erasable programmable read-Only register (Erasable Programmable Read Only Memory, EPROM), electrically erasable programmable read-Only Memory (EEPROM) flash Memory or other solid state Memory technology, CD-ROM, digital versatile disks (DIGITAL VERSATILE DISC, DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer storage medium is not limited to the one described above. The system memory 904 and mass storage device 907 described above may be collectively referred to as memory.

According to various embodiments of the disclosure, the computer device 900 may also operate by being connected to a remote computer on a network, such as the Internet. I.e., the computer device 900 may be connected to the network 912 through a network interface unit 911 coupled to the system bus 905, or other types of networks or remote computer systems (not shown) may be coupled using the network interface unit 911.

The memory further includes at least one instruction, at least one program, a code set, or an instruction set, where the at least one instruction, the at least one program, the code set, or the instruction set is stored in the memory, and the central processor 901 implements all or part of the steps in the calibration method for the exposable information ordering model shown in the above embodiments by executing the at least one instruction, the at least one program, the code set, or the instruction set.

Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

It is to be understood that the application is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

1. An information display method, characterized in that the method comprises:

The method comprises the steps of obtaining actual display results of each piece of displayable information, wherein the displayable information is obtained by predicting the prediction probability of receiving a specified user operation through a displayable information sorting model and selecting information to be displayed according to the predicted prediction probability, the displayable information sorting model comprises at least two cascaded sub-sorting models, the actual display results refer to the actual data of the specified user operation received after each piece of displayable information is displayed in a display platform, and the at least two sub-sorting models comprise a rough sorting model and a carefully selected sorting model cascaded behind the rough sorting model;

based on the actual display result of each displayable information and the prediction display result of each displayable information corresponding to the carefully chosen sorting model, acquiring the calibration coefficient of each displayable information corresponding to the carefully chosen sorting model;

Based on the prediction display results of the selection ordering model corresponding to the displayable information respectively and the prediction display results of the rough selection ordering model corresponding to the displayable information respectively, obtaining the calibration coefficients of the rough selection ordering model corresponding to the displayable information respectively;

Correcting the prediction probability of each displayable information of the at least two sub-ordering models based on the calibration coefficients of the at least two sub-ordering models;

and calculating the thousand cost of each piece of displayable information based on the corrected prediction probability of each piece of displayable information, and sequencing the thousand cost so as to display each piece of displayable information.

2. The method of claim 1, wherein the obtaining calibration coefficients for each of the presentable information corresponding to the pickorder model based on the actual presentation results for each of the presentable information and the predicted presentation results for each of the presentable information corresponding to the pickorder model, respectively, comprises:

Based on the actual display result of each displayable information and the predicted display result of each displayable information corresponding to the carefully chosen sorting model, acquiring a first calibration coefficient initial value of the carefully chosen sorting model and a calibration coefficient target value of each displayable information corresponding to the carefully chosen sorting model;

And acquiring the calibration coefficient of each displayable information corresponding to the carefully chosen sorting model based on the first calibration coefficient initial value and the calibration coefficient target value of each displayable information corresponding to the carefully chosen sorting model.

3. The method of claim 2, wherein the obtaining a first calibration coefficient initial value for the picklist model based on the actual presentation result of each presentable information and the predicted presentation result of each presentable information corresponding to the picklist model, respectively, and the calibration coefficient target value for each presentable information corresponding to the picklist model, respectively, comprises:

Acquiring a first calibration coefficient initial value of the carefully chosen sequencing model based on actual display results of the various pieces of displayable information in a first time period and prediction display results of the carefully chosen sequencing model corresponding to the various pieces of displayable information in the first time period respectively;

and acquiring calibration coefficient target values of the selected sorting models corresponding to the respective displayable information respectively based on the actual display results of the respective displayable information in the second time period.

4. The method of claim 3, wherein the actual display results comprise an actual exposure number, an actual click number, and an actual conversion number, wherein the predicted display results comprise a predicted click rate and a predicted conversion rate, and wherein the calibration coefficients comprise a click rate calibration coefficient and a conversion rate calibration coefficient;

the obtaining a first calibration coefficient initial value of the carefully chosen ranking model based on the actual display result of each piece of displayable information in the first time period and the predicted display result of each piece of displayable information corresponding to the carefully chosen ranking model in the first time period respectively includes:

5. The method of claim 3, wherein the actual display results comprise an actual exposure number, an actual click number, and an actual conversion number, wherein the predicted display results comprise a predicted click rate and a predicted conversion rate, and wherein the calibration coefficients comprise a click rate calibration coefficient and a conversion rate calibration coefficient;

The obtaining, based on the actual display result of each displayable information in the second time period, the calibration coefficient target value of each displayable information corresponding to the carefully selected sorting model, includes:

6. The method of claim 2, wherein the obtaining calibration coefficients for each presentable information corresponding to the pickorder model based on the first calibration coefficient initial value and the calibration coefficient target value for each presentable information corresponding to the pickorder model, respectively, comprises:

And carrying out weighted summation on the initial value of the first calibration coefficient and the calibration coefficient target value of the selected sorting model corresponding to each piece of displayable information respectively to obtain the calibration coefficient of the selected sorting model corresponding to each piece of displayable information.

7. The method of claim 1, wherein the obtaining calibration coefficients of the respective presentable information corresponding to the roughing ranking model based on the respective presentable information corresponding to the predicted presentation result of the carefully selected ranking model and the respective presentable information corresponding to the roughing ranking model, respectively, comprises:

Acquiring a second calibration coefficient initial value of the roughing sorting model based on the prediction display result of the selection sorting model corresponding to each displayable information and the prediction display result of the roughing sorting model corresponding to each displayable information, and the calibration coefficient target value of the roughing sorting model corresponding to each displayable information;

and based on the initial value of the second calibration coefficient and the calibration coefficient target value of the roughing and sorting model corresponding to each piece of displayable information, acquiring the calibration coefficient of the roughing and sorting model corresponding to each piece of displayable information.

8. The method of claim 7, wherein the obtaining the initial value of the second calibration coefficient of the roughing ranking model based on the predicted presentation results of the respective presentable information corresponding to the refined ranking model and the predicted presentation results of the respective presentable information corresponding to the roughing ranking model, respectively, and the calibration coefficient target value of the respective presentable information corresponding to the roughing ranking model, respectively, comprises:

acquiring a second calibration coefficient initial value of the rough sorting model based on the prediction display result of each displayable information corresponding to the carefully sorted sorting model in a third time period and the prediction display result of each displayable information corresponding to the rough sorting model in the third time period;

And acquiring calibration coefficient target values of the rough sorting model corresponding to the respective displayable information based on the prediction display results of the respective displayable information corresponding to the carefully sorted sorting model in a fourth time period.

9. The method of claim 8, wherein the predicted presentation results comprise a predicted click rate and a predicted conversion rate, and wherein the calibration coefficients comprise a click rate calibration coefficient and a conversion rate calibration coefficient;

The obtaining, based on the predicted display results of the respective displayable information corresponding to the selected ranking model in a third time period and the predicted display results of the respective displayable information corresponding to the rough ranking model in the third time period, a second calibration coefficient initial value of the rough ranking model includes:

Acquiring a sum of second estimated click numbers of each displayable information in the third time period and a sum of second estimated conversion numbers of each displayable information in the third time period, which correspond to the carefully chosen sorting model, based on the actual exposure numbers of each displayable information in the third time period and the estimated click rates and the estimated conversion rates of each displayable information corresponding to the carefully chosen sorting model in the third time period respectively;

10. The method of claim 8, wherein the predicted presentation results comprise a predicted click rate and a predicted conversion rate, and wherein the calibration coefficients comprise a click rate calibration coefficient and a conversion rate calibration coefficient;

The obtaining, based on the predicted display results of the respective displayable information corresponding to the selected sorting model in the fourth time period, calibration coefficient target values of the respective displayable information corresponding to the rough sorting model respectively includes:

Based on the actual exposure number of each piece of displayable information in the fourth time period, and the estimated click rate and the estimated conversion rate of each piece of displayable information corresponding to the carefully chosen sorting model in the fourth time period, acquiring the estimated click number and the estimated conversion number of each piece of displayable information in the fourth time period corresponding to the carefully chosen sorting model;

11. The method of claim 7, wherein the obtaining the calibration coefficients for the respective presentable information corresponding to the rougher ordering model based on the second calibration coefficient initial value and the calibration coefficient target values for the respective presentable information corresponding to the rougher ordering model comprises:

And carrying out weighted summation on the initial value of the second calibration coefficient and the calibration coefficient target value of the rough sorting model corresponding to each piece of displayable information respectively to obtain the calibration coefficient of the rough sorting model corresponding to each piece of displayable information.

12. An information presentation apparatus, the apparatus comprising:

the system comprises a first acquisition module, a first display module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring an actual display result of each piece of displayable information, the displayable information is the information which is predicted to receive the predicted probability of the operation of a specified user through a displayable information sorting model and is selected to be displayed according to the predicted probability obtained by prediction;

The third acquisition module is used for acquiring calibration coefficients of the selected sorting models corresponding to the respective displayable information based on the actual display results of the respective displayable information and the predicted display results of the selected sorting models corresponding to the respective displayable information; based on the prediction display results of the selection ordering model corresponding to the displayable information respectively and the prediction display results of the rough selection ordering model corresponding to the displayable information respectively, obtaining the calibration coefficients of the rough selection ordering model corresponding to the displayable information respectively;

The display module is used for correcting the prediction probability of each displayable information of the at least two sub-sequencing models based on the calibration coefficients of the at least two sub-sequencing models, calculating the thousand cost of each displayable information based on the corrected prediction probability of each displayable information and sequencing the corrected thousand cost of each displayable information so as to display each displayable information.

13. A computer device comprising a processor and a memory having stored therein at least one instruction, at least one program, code set or instruction set, the at least one instruction, at least one program, code set or instruction set being loaded and executed by the processor to implement the information presentation method of any of claims 1 to 11.

14. A computer readable storage medium having stored therein at least one instruction, at least one program, code set, or instruction set, the at least one instruction, the at least one program, the code set, or instruction set being loaded and executed by a processor to implement the information presentation method of any of claims 1 to 11.