JP7290272B2 - Ability measuring device, program and method - Google Patents

Description

Article 30, Paragraph 2 of the Patent Law applies September 3, 2018 Announced at the 46th Annual Meeting of the Japan Society of Behaviormetrics Abstracts

Application of Article 30, Paragraph 2 of the Patent Act September 4, 2018 Announced at the 46th Annual Meeting of the Japan Behaviormetric Society

Application of Article 30, Paragraph 2 of the Patent Law Published on September 5, 2018 at the 16th Conference of the Japan Society for Testing, Abstracts of Presentations, pp. 48 and 49

Application of Article 30, Paragraph 2 of the Patent Law September 9, 2018 Announced at the 16th Annual Meeting of the Japan Testing Society

The present invention relates to an ability measuring apparatus, program, and method for measuring multiple abilities of examinees.

Computerized adaptive testing (CAT) is an example of computer-based testing (CBT).

The adaptive test uses item response theory (IRT) to select appropriate items according to the test subject's response pattern, and the selected items are presented to the test subject.

With adaptive testing, test scoring is instantaneous and automatic, and different tests can be graded on the same scale. In addition, the adaptive test can measure the examinee's ability with a smaller number of items than paper-based testing (PBT).

Joel M. Brown & David J. Weiss(1977). An Adaptive Testing Strategy for Achievement Test Batteries. Research Report 77-6. Minnesota Univ., Minneapolis. David J. Weiss(2004). Computerized Adaptive Testing for Effective and Efficient Measurement in Counseling and Education. Measurement and Evaluation in Counseling and Development, 37(2) 70-84. Daniel O. Segal (1996). Multidimensional Adaptive Testing. Psychometrika, 61(2), 331-354. van der Linden, W. J.(1999). Multidimensional adaptive testing with a minimum errorvariance criterion, Journal of Educational and Behavioral Statistics, 24(4), 398-412. Mulder, J., & van der Linden, W. J. (2009). Multidimensional adaptive testing with optimal design criteria for item selection, Psychometrika, 74(2), 273-296.

JP-A-2004-46255 JP-A-2005-215023

Reduce the number of items presented to test-takers and the time required to collect responses when multiple items are presented to test-takers for each of multiple competencies, responses are collected, and multiple competencies are measured. , it is desirable to perform measurements efficiently and accurately.

The present invention has been made in view of the above circumstances, and provides an ability measuring apparatus, program, and method for efficiently and accurately measuring a plurality of abilities.

According to an embodiment of the invention, a performance measuring device includes a storage device and a processing device. A storage device stores item data including a plurality of items used for measuring a plurality of abilities. The processing unit uses the item data to perform adaptive tests for each of the plurality of competencies. The plurality of competencies includes a first competencies, a second competencies measured after the first competencies, and a third competencies measured after the second competencies. The processor selects initial items for a second adaptive test for a second ability based on a first final estimate of the first adaptive test for the first ability, and performs a second adaptive test. determine the prior distribution used in the proficiency estimation of , select initial items for a third adaptive test for the third proficiency based on the second final estimate of the second adaptive test, and select a third Determine the priors used in adaptive test ability estimation. The processing unit calculates an initial estimate for the second adaptive test based on the first final estimate and the first regression equation, and calculates a first estimate based on the initial estimate for the second adaptive test. Select the initial item of the second adaptive test, calculate the initial estimate of the third adaptive test based on the second final estimate and the second regression equation, and calculate the initial estimate of the third adaptive test Based on the initial estimates, select initial items for the third adaptive test. The processing device calculates the prior distribution with the mean value as the initial estimated value of the second adaptive test and the standard deviation as the standard error of the prediction error of the objective variable in the first regression equation, and the ability of the second adaptive test. The prior distribution used in the estimation, the initial estimated value of the third adaptive test as the mean value, and the standard error of the prediction error of the objective variable in the second regression equation as the standard deviation is the prior distribution used in the third adaptive test Let it be the prior distribution used in proficiency estimation for the type test.

According to the present invention, it is possible to efficiently and accurately measure a plurality of abilities.

1 is a block diagram showing an example of a performance measuring device according to a first embodiment; FIG. 4 is a flowchart showing an example of ability measurement processing according to the first embodiment; 10 is a flowchart showing an example of ability measurement processing according to the second embodiment; A graph showing an example of an item characteristic curve.

Hereinafter, embodiments of the invention will be described with reference to the drawings. In the following description, the same or substantially the same parts are denoted by the same reference numerals and their descriptions are omitted or briefly described.

(First embodiment)
In order to measure a plurality of abilities (or characteristics) of an examinee, the ability measuring apparatus according to the present embodiment selects a plurality of items for each of the plurality of abilities, presents them to the examinee, and provides answers to the plurality of items. (or responses or answers) are collected.

In this embodiment, the result of the n-1th adaptive test (n-1th ability estimate value) executed n-1th (n is an integer of 2 or more) and the n-th executed If the correlation between the result of the nth adaptive test (estimated value of the nth ability) is stronger than a predetermined level, based on the result of the n−1th adaptive test, the nth adaptive test Select and determine the initial terms and priors used in the test.
Note that the ability measuring device according to the present embodiment can perform effective measurement if the correlation coefficient between the n−1th ability estimate value and the nth ability estimate value is not zero. . Usually, the two performance estimates are considered to be positively correlated. However, the ability measuring device according to the present embodiment may be applied when two ability estimates have a negative correlation.

In the specific example of this embodiment, the case of measuring three abilities will be described for the sake of simplification, but the same applies to the case of measuring two abilities and the case of measuring four or more abilities.

FIG. 1 is a block diagram showing an example of the configuration of a performance measuring device 1 according to this embodiment.

The ability measuring device 1 measures the first to third abilities of an examinee. In this embodiment, the first ability is measured first, the second ability is measured after the first ability, and the third ability is measured after the second ability.

The ability measuring device 1 performs a first adaptive test for a first ability. In a first adaptive test, a first initial item is selected and a first prior distribution is used in ability estimation.

The ability measuring device 1 selects a second initial item of a second adaptive test for a second ability based on a first final estimate of the first adaptive test for the first ability, Determine a second prior distribution to be used in performance estimation for the adaptive test of 2. A second adaptive test uses a second initial item and a second prior distribution in ability estimation.

The ability measuring device 1 selects a third initial item of a third adaptive test for a third ability based on a second final estimate of the second adaptive test for the second ability, Determine the third prior distribution used in performance estimation for the adaptive test of 3. A third adaptive test uses a third initial term and a third prior distribution in ability estimation.

The ability measuring device 1 includes an input device 2, an output device 3, a storage device 4, and a processing device 5, for example. The input device 2, the output device 3, the storage device 4, and the processing device 5 can transmit and receive data to and from each other via a bus 6, for example. The ability measuring device 1 may be implemented by a plurality of information processing devices working together. For example, the performance measuring device 1 may be constructed by a network system composed of a client and a server.

The input device 2 is, for example, a keyboard, a pointing device, a receiving device, etc., and accepts or receives various commands, instructions, responses from examinees, and the like.

The output device 3 is, for example, a display, a speaker, a transmission device, or the like, and displays, outputs sound, or transmits various data such as measurement results of multiple abilities of the test subject.

The storage device 4 stores various data necessary for various processes executed by the processing device 5 . In the example of this embodiment, the storage device 4 includes, for example, a program 7, an item pool 8, answer data 9, estimated data 10, first to third ability measurement data 111 to 113, first and second regression models 121, 122, initial item data 13a, 13b, prior distribution data 14a, 14b, etc. are stored. Various data stored in the storage device 4 may be divided or combined as appropriate. Specifically, for example, the estimated data 10 and the first to third ability measurement data 111 to 113 may be combined. Storage device 4 may include, for example, non-volatile storage devices, and may include temporary storage media such as cache memory, for example.

The program 7, when executed by the processing device 5, realizes a function for efficiently and accurately measuring a plurality of abilities of the examinee. The details of the ability measurement process according to the present embodiment executed by this program 7 will be described later.

The item pool 8 is data containing a set of items used in the first to third adaptive tests.

The answer data 9 includes the examinee's answers to each of the plurality of items presented to the examinee and correct/wrong data regarding the answers.

Estimated data 10 includes estimated values calculated by performance estimation in adaptive testing. In this embodiment, based on the estimated values contained in the estimated data 10, items to be presented next to the examinee are selected.

Each of the first through third ability measurement data 111-113 includes the final estimates calculated in the first through third adaptive tests for measuring the first through third abilities.

First regression model 121 includes formulas, data, or programs used to select a second initial item to be used in a second performance measure based on first performance measurement data 111 .

Further, the first regression model 121 may be a formula, data, or Including program.

More specifically, first regression model 121 includes a first regression equation for selecting a second initial item based on first performance measurement data 111 . The first regression formula uses the ability value of the first ability as an explanatory variable and the ability value of the second ability as an objective variable. An initial estimate of the second ability is calculated by substituting the final estimate of the first ability into the first regression equation, and a second initial item corresponding to the initial estimate of the second ability is selected.

In addition, the first regression model 121 has the information for determining the prior distribution with the initial estimated value of the second ability as the mean and the standard error of the prediction error of the objective variable in the first regression equation as the standard deviation. include.

Second regression model 122 includes formulas, data, or programs used to select a third initial item for use in a third performance measure based on second performance measurement data 112 .

Further, the second regression model 122 may be a formula, data, or Including program.

More specifically, second regression model 122 includes a second regression equation for selecting a third initial item based on second performance measurement data 112 . The second regression formula uses the ability value of the second ability as an explanatory variable and the ability value of the third ability as an objective variable. By substituting the final estimate of the second ability into the second regression equation, the initial estimate of the third ability is calculated, and the third initial item corresponding to the initial estimate of the third ability is selected.

In addition, the second regression model 122 has information for determining a prior distribution with the initial estimated value of the third ability as the mean and the standard error of the prediction error of the objective variable in the second regression equation as the standard deviation. include.

The initial item data 13a is data indicating the second initial items used in the second adaptive test.

The initial item data 13b is data indicating the third initial item used in the third adaptive test.

The prior distribution data 14a is data representing a second prior distribution used for ability estimation in the second adaptive test.

The prior distribution data 14b is data representing a third prior distribution used for ability estimation in the third adaptive test.

The processing device 5 executes the program 7 stored in the storage device 4, and functions as the control unit 15, the first to third ability measurement units 161 to 163, the initial item selection unit 17, and the prior distribution determination unit 18. . Note that the control unit 15, the first to third ability measurement units 161 to 163, and the initial item selection unit 17 may be divided or combined as appropriate. Specifically, for example, the first to third ability measuring units 161 to 163 may be a common ability measuring unit that can measure the first ability to the third ability. For example, the initial item selection unit 17 may be divided into a portion for selecting the second initial item and a portion for selecting the third initial item. For example, the prior distribution determination unit 18 may be divided into a part that determines the second prior distribution and a part that determines the third prior distribution.

The processing device 5 may be, for example, a processor such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or a DSP (Digital Signal Processor).

The control unit 15 may be implemented by, for example, an operating system. The first to third ability measuring units 161 to 163, the initial item selecting unit 17, and the prior distribution determining unit 18 may be realized by, for example, application programs running on the operating system.

The control unit 15 performs control for inputting the examinee's answers and the like from the input device 2 . Further, the control unit 15 performs control for reading the program 7 and various data stored in the storage device 4 and performs control for storing various data in the storage device 4 . Further, the control unit 15 controls the output device 3 to output various data. In the following description, the control of the input device 2, the output device 3, and the storage device 4 by the control unit 12 is omitted in order to simplify the description.

In this embodiment, the correlation coefficient between the estimated value of the first ability and the estimated value of the second ability is greater than a predetermined level (threshold), and the correlation between the first ability and the second ability is It is confirmed in advance that the correlation is strong, the correlation coefficient between the estimated value of the second ability and the estimated value of the third ability is greater than a predetermined level, and the correlation between the second ability and the third ability is It is assumed that it is confirmed in advance that the correlation between is strong. Note that the correlation coefficient between the first ability estimate and the second ability estimate need not be zero, and similarly, the second ability estimate and the third ability estimate The correlation coefficient between should be non-zero.

The first ability measuring section 161 executes the first adaptive test and measures the first ability of the examinee. The first ability measurement unit 161 selects an item from, for example, an item pool 8 stored in the storage device 4, presents the selected item to the examinee using the output device 3, and An answer is accepted using the input device 2, and the answer data 9 including the answer and correct/wrong data is stored in the storage device 4. FIG. The first initial item used in the measurement of the first ability and the first prior distribution used in the ability estimation at the time of the measurement of the first ability may be set according to a predetermined rule, for example. , may be preset from experimental data. The first ability measurement unit 161 causes the storage device 4 to store the first ability measurement data 111 including the final estimated value of the examinee's first ability.

The initial item selection unit 17 selects a second initial item to be used in the second adaptive test based on the first ability measurement data 111 and the first regression model 121 stored in the storage device 4. The initial item data 13 a indicating the second initial item is stored in the storage device 4 .

Based on the first ability measurement data 111 and the first regression model 121 stored in the storage device 4, the prior distribution determination unit 18 determines the second data used in ability estimation in the second adaptive test. A prior distribution is determined, and prior distribution data 14 a indicating the determined second prior distribution is stored in the storage device 4 .

The second ability measuring unit 162 executes the second adaptive test based on, for example, the initial item data 13a and the prior distribution data 14a stored in the storage device 4, and measures the second ability of the examinee. take measurements. For example, the second ability measurement unit 162 selects an item from the item pool 8 stored in the storage device 4, presents the selected item to the examinee using the output device 3, and An answer is accepted using the input device 2, and the answer data 9 including the answer and correct/wrong data is stored in the storage device 4. FIG. The second ability measurement unit 162 causes the storage device 4 to store the second ability measurement data 112 including the final estimated value of the second ability of the examinee.

The initial item selection unit 17 selects a third initial item to be used in the third adaptive test based on the second ability measurement data 112 and the second regression model 122 stored in the storage device 4. The initial item data 13b indicating the third initial item is stored in the storage device 4.

Based on the second ability measurement data 112 and the second regression model 122 stored in the storage device 4, the prior distribution determination unit 18 determines a third ability estimation used in ability estimation in the third adaptive test. A prior distribution is determined, and prior distribution data 14b representing the determined third prior distribution is stored in the storage device 4. FIG.

The third ability measuring unit 163 executes the third adaptive test based on, for example, the initial item data 13b and the prior distribution data 14b stored in the storage device 4, and measures the third ability of the examinee. take measurements. The third ability measurement unit 163 selects an item from, for example, an item pool 8 stored in the storage device 4, presents the selected item to the examinee using the output device 3, and An answer is accepted using the input device 2, and the answer data 9 including the answer and correct/wrong data is stored in the storage device 4. FIG. The third ability measurement unit 163 causes the storage device 4 to store the third ability measurement data 113 including the final estimated value of the examinee's third ability.

In the embodiment described above, the nth initial item and the nth prior distribution used in the nth adaptive test are selected and determined based on the result of the n-1th adaptive test.

FIG. 2 is a flow chart showing the processing from the measurement of the first ability to the measurement of the third ability by the ability measuring device 1 according to this embodiment.

In step S201, the first ability measuring unit 161 selects a first initial item from the item pool 8 stored in the storage device 4 according to a rule set in advance by experiment or the like.

In step S202, the first ability measuring unit 161 causes the output device 3 to output the first initial item. This presents the examinee with the first initial item.

In step S<b>203 , the first ability measuring unit 161 receives the examinee's answers to the first initial items from the input device 2 , and stores the answer data 9 including the answers and correct/wrong data in the storage device 4 .

In step S204, the first ability measuring unit 161 estimates the examinee's ability based on the answer data 9 stored in the storage device 4 and the first prior distribution preset by experiment or the like, Estimated data 10 including the calculated estimated value is stored in the storage device 4 .

In step S<b>205 , the first ability measuring unit 161 selects the next item from the item pool 8 stored in the storage device 4 based on the estimated data 10 stored in the storage device 4 .

In step S206, the first ability measuring unit 161 causes the output device 3 to output the next selected item.

In step S207, the first ability measuring unit 161 receives the examinee's answers to the items from the input device 2, and updates the answer data 9 in the storage device 4 so that the accepted answers and correct/wrong data are added.

In step S208, the first ability measuring unit 161 estimates the examinee's ability based on the answer data 9 stored in the storage device 4 and the first prior distribution, and stores the calculated estimated value. Update the estimated data 10 stored in the device 4 .

The above steps S205 to S208 are repeatedly executed until the conditions for ending the first adaptive test are satisfied. For example, if the accuracy of the estimate obtained by the ability estimation in the first adaptive test is equal to or greater than a predetermined level (if the standard error is less than a predetermined level), or When the number of items obtained exceeds a predetermined number, it is determined that the first adaptive test is finished.

If the first adaptive test satisfies the end condition, in step S209 the first ability measurement unit 161 stores the first ability measurement data 111 including the final estimated value of ability estimation in the first adaptive test. stored in the device 4;

In step S210, the initial item selection unit 17 selects the second item used in the second adaptive test based on the first regression model 121 and the first ability measurement data 111 stored in the storage device 4. An initial item is selected from the item pool 8 stored in the storage device 4 . Then, the initial item selection unit 17 causes the storage device 4 to store the initial item data 13a indicating the second initial item. Also, the prior distribution determination unit 18 determines the second model used in the ability estimation in the second adaptive test based on the first regression model 121 and the first ability measurement data 111 stored in the storage device 4 . 2 priors are determined. Then, the prior distribution determination unit 18 causes the storage device 4 to store the prior distribution data 14a indicating the determined second prior distribution.

In step S<b>211 , the second ability measuring unit 162 causes the output device 3 to output the second initial item indicated by the initial item data 13 a stored in the storage device 4 . This presents the examinee with the second initial item.

In step S212, the second ability measurement unit 162 receives the examinee's answers to the second initial items from the input device 2, and stores the answer data 9 in the storage device 4 so that the accepted answers and correct/wrong data are added. Update.

In step S213, the second ability measuring unit 162 estimates the examinee's ability based on the answer data 9 stored in the storage device 4 and the second prior distribution indicated by the prior distribution data 14a, and calculates The estimated data 10 stored in the storage device 4 is updated with the estimated value obtained.

In step S<b>214 , the second ability measuring unit 162 selects the next item from the item pool 8 stored in the storage device 4 based on the estimated data 10 stored in the storage device 4 .

In step S215, the second ability measuring unit 162 causes the output device 3 to output the next selected item.

In step S216, the second ability measuring unit 162 receives the examinee's answers to the items from the input device 2, and updates the answer data 9 in the storage device 4 so that the accepted answers and correct/wrong data are added.

In step S217, the second ability measuring unit 162 estimates the examinee's ability based on the answer data 9 stored in the storage device 4 and the second prior distribution, and stores the calculated estimated value. Update the estimated data 10 stored in the device 4 .

The above steps S214 to S217 are repeatedly executed until the end condition of the second adaptive test is satisfied.

If the second adaptive test satisfies the termination condition, in step S218 the second ability measurement unit 162 stores the second ability measurement data 112 including the final estimated value of the ability estimation in the second adaptive test. stored in the device 4;

In step S219, the initial item selection unit 17 selects the third item used in the third adaptive test based on the second regression model 122 and the second ability measurement data 112 stored in the storage device 4. An initial item is selected from the item pool 8 stored in the storage device 4 . Then, the initial item selection unit 17 causes the storage device 4 to store the initial item data 13b indicating the third initial item. Also, the prior distribution determination unit 18 determines the second regression model 122 and the second ability measurement data 112 stored in the storage device 4, which are used for ability estimation in the third adaptive test. 3 priors are determined. Then, the prior distribution determination unit 18 causes the storage device 4 to store the prior distribution data 14b indicating the determined third prior distribution.

In step S<b>220 , the third ability measuring unit 163 causes the output device 3 to output the third initial item indicated by the initial item data 13 b stored in the storage device 4 . This presents the examinee with the third initial item.

In step S221, the third ability measurement unit 163 receives the examinee's answers to the third initial item from the input device 2, and stores the answer data 9 in the storage device 4 so that the accepted answers and correct/wrong data are added. Update.

In step S222, the third ability measuring unit 163 estimates the examinee's ability based on the answer data 9 stored in the storage device 4 and the third prior distribution indicated by the prior distribution data 14b, and calculates The estimated data 10 stored in the storage device 4 is updated with the estimated value obtained.

In step S<b>223 , the third ability measuring unit 163 selects the next item from the item pool 8 stored in the storage device 4 based on the estimated data 10 stored in the storage device 4 .

In step S224, the third ability measuring unit 163 causes the output device 3 to output the next selected item.

In step S225, the third ability measuring unit 163 receives the examinee's answers to the items from the input device 2, and updates the answer data 9 in the storage device 4 so that the accepted answers and correct/wrong data are added.

In step S226, the third ability measuring unit 163 estimates the examinee's ability based on the answer data 9 stored in the storage device 4 and the third prior distribution, and stores the calculated estimated value. Update the estimated data 10 stored in the device 4 .

The above steps S223 to S226 are repeatedly executed until the conditions for ending the third adaptive test are satisfied.

If the third adaptive test satisfies the termination condition, in step S227, third ability measurement unit 163 stores third ability measurement data 113 including the final estimated value of ability estimation in the third adaptive test. stored in the device 4;

In the embodiment described above, the initial items and prior distributions used in the nth adaptive test are selected and determined based on the results of the n-1th adaptive test.

As a result, the number of items presented to examinees of multiple adaptive tests relating to multiple abilities can be reduced, and multiple abilities can be accurately measured in a short period of time.

The adaptive test performed in this embodiment is an application of the one-dimensional adaptive test, and is different from the multidimensional adaptive test. Therefore, multiple ability measurement results are obtained with a smaller amount of calculation than the multidimensional adaptive test. be able to. In addition, since it is easy to recognize whether the first adaptive test, the second adaptive test, or the third adaptive test is being performed, explanations and exercises should be provided before measuring each ability. It can be carried out.

Effects obtained by this embodiment will be described in more detail.

The adaptive test flow is as follows.

As a first step, an initial item (question) is presented to the examinee. As initial items, for example, about three items such as a difficult item, an intermediate item, and an easy item may be presented.

As a second step, the ability of the examinee is estimated by Bayesian estimation based on the correct/wrong data of the answers to the items presented to the examinee so far, and an estimated value is calculated. A standard normal distribution, for example, is used as a prior distribution necessary for calculating this estimated value.

As a third step, the adaptive test is terminated if the standard error of the calculated estimate is less than a predetermined level and the estimate is determined to be accurate. If the standard error of the calculated estimated value is equal to or greater than a predetermined level, an item appropriate for the current estimated value of the examinee is selected from the item pool 8 and presented to the examinee, and the second step is performed. return.

In this embodiment, the first and second steps are statistically improved in the flow when the adaptive test as described above is used continuously for a plurality of tests.

For the first step, the final estimate of the first adaptive test is used to statistically predict the initial estimate in a second adaptive test performed after the first adaptive test, predicting It has been refined to select an appropriate initial term for the initial guess made.

For the second step, let the statistically predicted initial estimate in the second adaptive test and the standard error of the prediction error of the objective variable in the first regression equation be the mean and standard deviation, respectively. It is refined to determine the prior distribution and use it for performance estimation in the second adaptive test.

This allows the test taker to take the second adaptive test with appropriate initial items and prior distributions.

Adaptive tests have the advantage of shortening the time required to measure abilities, but if multiple abilities are measured sequentially by multiple adaptive tests, the total time taken for the tests may become longer. be. As a method for shortening the examination time, there is a method according to Non-Patent Document 1 described above. This non-patent document 1 method improves on the first step in adaptive testing. The present embodiment achieves a shorter test time than the method of Non-Patent Document 1 and does not lower the accuracy.

In addition, there is a multidimensional adaptive test as a method for shortening the time taken for the adaptive test. This multi-dimensional adaptive test does not determine which ability of the plurality of abilities the item presented to the examinee next measures. For this reason, the multidimensional adaptive test is difficult to use in cases where an example is provided for each ability measurement. In this embodiment, adaptive tests are conducted for each ability for each of a plurality of abilities. It is easy to grasp which item is the item for measuring ability.

In this embodiment, since the adaptive test can be conducted within a limited time such as class time, it is possible to conduct a questionnaire survey together with the adaptive test.

Table 1 shows comparative examples of simulation results for the normal adaptive test based on item response theory, the method of Non-Patent Document 1, the multidimensional adaptive test, and the ability measurement process according to the present embodiment.

Table 1 shows that 1,000 virtual examinees measured ability by four methods: a normal adaptive test, the method of Non-Patent Document 1, a multidimensional adaptive test, and ability measurement processing according to this embodiment. It represents the statistics on the number of responses for each examinee when

The number of abilities measured is three, and the correlation coefficients between abilities are all 0.5.

Also, in the simulations in Table 1, performance scores were generated from a standard normal distribution with a variance of one. A two-parameter logistic model was used for item response theory. The termination condition for the adaptive test was when the standard deviation of the estimated values was 0.35 or less. Discrimination power was generated from a uniform distribution from 0.5 to 1.5. Difficulty was generated from a uniform distribution of -4 to 4.

From Table 1, it is preferable that the average number of responses is small in the ability measurement process according to this embodiment and the multidimensional adaptive test. The number of responses corresponds to the number of items required for ability measurement. Comparing the ability measurement process according to this embodiment with the multidimensional adaptive test, the maximum value of the number of responses in the ability measurement process according to this embodiment and the standard deviation (variation) of the number of responses between examinees are Smaller than the maximum number of responses in a multidimensional adaptive test and the standard deviation of the number of responses per test taker.

Table 2 shows comparative examples of the evaluation results of the normal adaptive test, the method of Non-Patent Document 1, the multidimensional adaptive test, and the ability measurement processing according to this embodiment.

As shown in Tables 1 and 2, the ability measurement process according to the present embodiment requires a small number of responses, the maximum number of responses is the smallest, and the standard deviation of the number of responses among examinees is small. The ability measurement process according to this embodiment is also effective in actual measurement.

In this embodiment, an example is described in which the n-1th adaptive test and the nth adaptive test for measuring abilities that are highly correlated with each other are successively executed. However, between the n-1th adaptive test and the nth adaptive test, adaptive tests for other capabilities may be performed.

(Second embodiment)
In the first embodiment above, the initial terms and priors used in the nth adaptive test are selected and determined based on the final estimates of the n-1th adaptive test.

On the other hand, in the present embodiment, the m-2th adaptive test that has a strong correlation with the m-th (m is an integer equal to or greater than 3) executed m-th adaptive test And if the m-1th adaptive test executed m-1th is performed before the m-th adaptive test, the result of the m-2th adaptive test (m-th 2 ability estimate) and the results of the m-1th adaptive test (m-1th ability estimate), select and decide.

FIG. 3 is a flowchart showing an example of ability measurement processing according to this embodiment.

In this embodiment, it is assumed that the third ability has a stronger correlation than the predetermined level with the first and second abilities. However, in the ability measuring device according to the present embodiment, the correlation coefficient between the first estimated ability value and the third ability estimated value is not zero, and furthermore, the second ability estimated value and the second ability estimated value A valid measurement can be made if the correlation coefficient between the 3 capacity estimates is non-zero.

In FIG. 3, step S300 is different from step S219 in FIG. 2, but other steps in FIG. 3 are the same as those in FIG.

In step S300, the initial item selection unit 17 selects the third adaptive A third initial item to be used in the type test is selected from the item pool 8 stored in the storage device 4 . Then, the initial item selection unit 17 causes the storage device 4 to store the initial item data 13b indicating the third initial item.

Further, the prior distribution determination unit 18 performs a third adaptive test based on the second regression model 122, the first ability measurement data 111, and the second ability measurement data 112 stored in the storage device 4. Determine a third prior distribution to be used in the ability estimation in . Then, the prior distribution determination unit 18 causes the storage device 4 to store the prior distribution data 14b indicating the determined third prior distribution.

In this embodiment, the second regression model 122 includes multiple regression equations for selecting the third initial item based on the first performance measurement data 111 and the second performance measurement data 112. FIG. The multiple regression formula uses the ability value of the first ability as the first explanatory variable, the ability value of the second ability as the second explanatory variable, and the ability value of the third ability as the objective variable. By substituting the final estimated value of the first ability and the final estimated value of the second ability into the multiple regression equation, an initial estimated value of the third ability is calculated, corresponding to the initial estimated value of the third ability A third initial item is selected.

Further, in the present embodiment, the second regression model 122, for example, assumes that the initial estimated value of the third ability is the average in the prior distribution, and the standard error when obtaining the prediction interval of the objective variable in the multiple regression equation is the standard deviation and

As described above, when multiple adaptive tests are performed, the results of the previously measured multiple abilities are used to select and determine the initial items and prior distribution of the adaptive test to be performed next. By doing so, it is possible to efficiently and accurately measure a plurality of abilities of the examinee.

In the present embodiment, the case where the m-2th adaptive test, the m-1th adaptive test, and the m-th adaptive test for measuring abilities with a strong correlation are successively executed is assumed. It is described as an example. However, between the m-2th adaptive test and the m-1th adaptive test, adaptive tests of other capabilities may be performed, such that the m-1th adaptive test and the m-th adaptive test In between adaptive tests, adaptive tests of other capabilities may be performed.

Also, the initial items and prior distribution of the m-th adaptive test are selected and determined from the results of two adaptive tests, but may be selected and determined from the results of three or more adaptive tests.

(Third embodiment)
In this embodiment, the ability measurement processing executed by the ability measuring device 1 described in the first and second embodiments will be described in further detail.

<Item response theory model>
The ability measurement process according to the first and second embodiments uses item response theory. Item response theory makes it possible to separate item difficulty and examinee ability by using item characteristic functions.

The item characteristic function is a function of θ that indicates the probability that an examinee with ability θ will answer a certain item correctly. A curve expressing the item characteristic function, with the ability θ of the examinee on the horizontal axis and the probability of correct answer of the item on the vertical axis, is called the item characteristic curve. A logistic model is used as an example of an item response theory model that expresses an item characteristic curve. A two-parameter logistic model (PLM) will be described below as an example.

2PLM uses two parameters, a discrimination power parameter a and a difficulty parameter b. Expression (1) is obtained by expressing the correct answer probability of item j as a function of ability θ using discriminating power parameter a _j and difficulty parameter b _j .

In equation (1), the domain of θ is all real numbers.

FIG. 4 is a graph showing an example of item characteristic curves. In FIG. 4, the solid line indicates a=1.0 and b=-1.0, the dashed line indicates a=0.5 and b=-1.0, and the dotted line indicates a=1.0. Item characteristic curves for 0 and b=1.5 are shown, respectively. The discriminating power parameter a represents the slope of the curve when the correct answer probability is 0.5, and the difficulty parameter b represents the value of ability θ when the correct answer probability is 0.5. In general, the larger the value of a, the steeper the slope of the item characteristic curve, and the higher the distinguishing power. However, there is no item with high discriminative power for examinees of any ability, and it is suitable to measure the ability of examinees whose ability is near b, where the slope of the graph is steep.

For example, both the solid line graph and the dashed line graph show that the item is suitable for measuring examinees whose ability θ is around -1, but the solid line graph has a steeper slope than the dashed line graph. Therefore, it has high discriminative power. In addition, since the slope of the steepest part of the solid line graph and the dotted line graph are the same, the discriminative power is the same, but the ability θ suitable for measurement is different, and the dotted line graph has a higher ability to discriminate. Suitable for

<item pool>
When the same test is administered to multiple groups, it is rated as easy for high-skilled groups and difficult for low-skilled groups. In this way, the evaluation of the difficulty of a test depends on the ability of the examinee group to take the test, which is called sample dependence. In addition, individual examinees' test scores will be higher if the item is overall easier and lower if the item is overall more difficult. In this way, the fact that the evaluation of examinee's ability depends on the degree of difficulty of the item is called item dependency. By using the item response theory, it is possible to reduce specimen dependence and item dependence to evaluate examinee ability. One of the conditions for reducing sample dependence and item dependence is that the values of the discrimination power parameter and the difficulty parameter are already clarified for each item that constitutes the test, or that they are clarified after the fact It is necessary to become A set of items for which parameter estimates have already been obtained is called an item pool 8 . Estimates of parameters for an item require prior knowledge of how a certain number of test-takers, representative of the test-taker population for which the test is intended, responded to the item. The item can be used over and over again, and preferably the content of the test performed is not disclosed and kept confidential. In this embodiment, it is assumed that parameter estimates for items have already been obtained and an item pool 8 has been generated prior to execution of the first adaptive test.

<Estimation of ability parameters>
Methods of estimating ability parameters include maximum likelihood estimation, MAP estimation, and EAP estimation. The MAP estimation method and the EAP estimation method are estimation methods using the Bayesian method.

The maximum likelihood estimation method is a method of obtaining a value of θ that maximizes the probability (likelihood) of obtaining that reaction pattern when ability θ is fixed at a certain value. When an examinee with ability θ answers J items and obtains a reaction pattern u=(u _{1 ,} u ₂ , _. When the probability of incorrect answer q _j (θ)=1−p _j (θ), the likelihood function is expressed by equation (2).

Taking the natural logarithm In of both sides of this equation (2), differentiating with respect to θ to obtain θ such that the slope of the tangent becomes zero, and solving the equation that makes the equation zero, we obtain θ A maximum likelihood estimate is obtained. θ obtained from the equation (3) is the maximum likelihood estimation value.

This equation (3) is called a likelihood equation. In 2PLM, the result of differentiation of p _j (θ) and q _j (θ) with respect to θ is given by Equation (4).

A successive approximation method is used as a numerical calculation method for using the maximum likelihood estimation method. If all questions are answered correctly or incorrectly by the maximum likelihood estimation method, the maximum likelihood estimation value becomes infinity or minus infinity, respectively, and a solution cannot be obtained. As a countermeasure for such cases, the Bayesian estimation described below can be used.

The MAP estimation method is a method in which θ that maximizes the probability density of the posterior distribution is used as an estimated value. The posterior distribution is a probability distribution regarding θ, and the MAP estimation method obtains the value of θ with the highest probability by looking at the posterior distribution.

According to Bayes' theorem, the posterior distribution g(θ|u) is obtained from the likelihood function L(θ|u) and the prior distribution f(θ) according to Equation (5).

Since the denominator of equation (5) can be treated as a constant that does not include θ, g(θ|u) is considered to be proportional to L(θ|u)f(θ). Here, if the prior distribution f(θ) of θ is assumed to be a normal distribution N(μ, σ ² ) with mean μ and variance σ ² , (6) is obtained.

Taking the natural logarithm of both sides of the equation (6) and differentiating with respect to θ to zero, the equation (7) is obtained.

Solving equation (7) yields the MAP estimate. When solving the equation (7), the iterative approximation method is used as in the case of the maximum likelihood estimation method.

The EAP estimation method is an estimation method in which the expected value of the posterior distribution of θ is the estimated value of θ. The EAP estimator is expressed by Equation (8) as the expected value of Equation (5), which is the posterior distribution of θ.

Both the MAP estimation method and the EAP estimation method estimate θ based on the same posterior distribution.

<Selection of initial items and ability estimation in the first adaptive test>
In the first adaptive test, initial items are selected from the item pool 8 and presented to the test taker. The number of items selected as initial items may be one or plural. Assuming that the distribution of examinee's abilities is a standard normal distribution, an item should be selected so that the examinee's ability can be well measured near θ=0. If one item is selected as the initial item, select the item whose item difficulty parameter is closest to zero. In the first adaptive test in this embodiment, three items whose item difficulty parameters are closest to -1, 0, and 1 are selected as initial items. An initial estimate θh of the ability value θ is calculated from the answers to the selected initial items. For estimating θ, Bayesian estimation using an appropriate distribution as a prior distribution is used, but maximum likelihood estimation may be used.
Note that the method of selecting the initial items described above is an example, and can be changed as appropriate. For example, the items whose item difficulty parameters are closest to -0.5, 0, and 0.5, respectively, may be selected as initial items. For example, the two items whose item difficulty parameters are closest to −0.5 and 0.5, respectively, may be selected as initial items. Thus, the value of the item difficulty parameter used in selecting the initial items can be changed, and the number of items selected as the initial items can also be changed.

<Item selection criteria>
After the first ability value is estimated, selection of the next item and ability estimation are repeated. The following items are based on item selection criteria: Fisher's Information Criterion (MFI), Owen's Approximate Bayesian Criterion (Owen), Likelihood Weighted Information Criterion (MLWI), or Posterior Weighted Information Criterion (MPWI). selected.

The Fisher information amount is an amount of information representing the degree to which the logarithmic likelihood decreases as θ separates from the maximum likelihood estimator. The steeper the Fisher information amount, the better the estimation of the maximum likelihood estimator. For this reason, MFI is used to compare the information content of each item using the reduced state of the maximum likelihood estimator. Owen selects items for which the absolute value of the difference between the expected value of the posterior distribution of θ and the degree of difficulty in the item pool 8 is equal to or less than a certain value. Also, the estimated value θh of θ may have a large difference from θ especially at the beginning of the adaptive test. MFI and Owen, which are greatly affected by θh, may not be able to select appropriate items. make it possible. Furthermore, in MPWI, the posterior distribution is used for weighting, since more appropriate item selection is possible when an appropriate prior distribution is set for MLWI.

MFI is a method of selecting an item j that maximizes I(θh _j ) given by equation (10) below. The Fisher information I(θ) is the negative expected value of the value obtained by second-order differentiating the logarithmic likelihood of θ with respect to the reaction pattern with θ.

This formula (9) is the total amount of item information in formula (10), and MFI is a method that focuses on the value of the item information amount. However, q _j (θ)=1−p _j (θ).

The order of items to be selected is represented using k. MLWI is the integral value of the value obtained by weighting the Fisher information amount at each θ of item j by the likelihood L (θ|u _{j1 , .} , is the method of selecting the maximum item.

MPWI is a Bayesian MLWI, and is a method of selecting an item that maximizes the expression (12) with the prior distribution of θ as g(θ).

At the beginning of adaptive testing, the difference between θh and θ can be large. Even in such a situation, the MFI selects items that are highly dependent on items with a large amount of Fisher's information, and the Owen's method selects the item whose difficulty parameter is closest to θh. However, MLWI and MPWI differ from MFI and Owen in that they use a value weighted by the likelihood or prior distribution for the Fisher information amount.

As shown in equations (11) and (12), MLWI and MPWI differ in that the likelihood or the posterior distribution is used as the weight for the Fisher information amount.

In MLWI, at the beginning of the adaptive test, the items selected can be highly dependent on the discriminant power parameters. The reason for this is that the likelihood is flat at the beginning of the adaptive test, the estimated value of θ is almost indeterminate, and I _jk (θ) increases in proportion to a _j ² . Therefore, as a result, other conditions are not reflected so much, and items with large discrimination power parameters are likely to be selected.

In addition, since MPWI uses a prior distribution, the influence of the prior distribution is reflected even in the early stage of the adaptive test, and θ can be estimated appropriately to some extent even in the early stage of the adaptive test compared to MLWI. However, there is no problem if the prior distribution is set appropriately, but if the prior distribution is not appropriate, it is difficult to reduce the standard error of θh, and the number of items to be selected may increase.

<Completion condition of ability estimation>
Items after the initial item are selected by using the item selection criteria described above. Each time an item is selected and presented to the examinee and an answer is obtained, the estimated ability θh is updated from the answers of all the items selected so far.

The process from item selection to ability estimation is repeated until the end condition is satisfied, and when the end condition is satisfied, the final estimated value θh is obtained from the answers to all the selected items. The termination condition may be that the standard error of the estimated value of θh is equal to or less than a threshold, or that the number of selected items reaches a threshold, or a combination of these may be used for
In this embodiment, a termination condition is determined for each of a plurality of abilities.

<How to estimate ability>
In this embodiment, in the first adaptive test, processing similar to that of a normal adaptive test is executed. Then, in a second adaptive test performed after the first adaptive test, an initial An estimate is calculated and the initial item for the second adaptive test is selected using the calculated initial estimate. Also, a prior distribution is determined based on the initial estimates and their standard deviations, and in a second adaptive test, the determined prior distributions are used to calculate an estimate of ability by a Bayesian method.

<Selection of initial items according to the first embodiment>
In the first embodiment, a simple regression equation is used in which the ability value of the past examinee's first ability is an explanatory variable and the ability value of the past examinee's second ability is an objective variable. Here, a past examinee is a person whose multiple abilities have been measured. By substituting the final estimated value of the first ability of the current examinee into this simple regression equation as an explanatory variable, the initial estimated value θ ₀ of the second ability is calculated. The second adaptive test uses the initial term corresponding to this second initial estimate of ability θ ₀ .

In the first embodiment, when measuring the third ability, the ability value of the past examinee's second ability is an explanatory variable, and the ability value of the past examinee's third ability is an objective variable. A regression equation is used. The initial estimated value θ ₀ of the third ability is calculated by substituting the final estimated value of the second ability of the current examinee into this simple regression equation as an explanatory variable. The third adaptive test uses the initial item corresponding to the initial estimate θ ₀ of this third ability.

The same is true when measuring the fourth and subsequent abilities. Specifically, a simple regression formula is used in which the ability value of the past examinee's n-1th ability is an explanatory variable and the ability value of the past examinee's nth ability is an objective variable. The initial estimated value θ ₀ of the n-th ability is calculated by substituting the final estimated value of the examinee's n-1th ability into this simple regression equation as an explanatory variable. The nth adaptive test uses the initial item corresponding to this nth initial estimate of ability θ ₀ .

Each of the above simple regression equations is set in advance from actual measurement data of past examinees.

Below we describe four examples of selecting initial terms based on the initial estimate θ ₀ . Note that the four examples described below can be changed as appropriate.
In the first example, the three items whose difficulty parameters are closest to -1, 0, and 1 are selected.
In the second example, the three items whose difficulty parameters are closest to θh ₀ −0.5, 0, and θh ₀ +0.5 are selected.
In the third example, only the one item whose difficulty parameter is closest to θh ₀ is selected.
In the fourth example, only one item with the maximum Fisher information amount for θh ₀ is selected.

Of the above first to fourth examples, the first example does not use θh ₀ . This first example can be used, for example, when selecting initial items for a first adaptive test, or when there is no correlation between multiple competencies.

In the second example, the initial estimated value θh ₀ determined from the final estimated value of the adaptive test performed on the examinee is used as a reference, and three items close to this initial estimated value θh ₀ are selected. In this second example, the reason why the interval between the difficulty level parameters of the selected items is set to 0.5 is that, unlike the first example, θh ₀ is calculated and the error can be reduced. In a third example, only one item whose difficulty parameter is close to the initial estimate θh ₀ is selected. All the first to third examples are the selection of initial items based only on the difficulty parameter, but the fourth example is the selection of the initial items based on the Fisher information amount, and the discriminative power is selected in addition to the difficulty parameter. Parameters and likelihoods are used. The second to fourth examples are applied when there is a predetermined level or more of correlation between a plurality of abilities.

In each adaptive test, any of the first to fourth examples can be applied to the initial item selection, and the item selection criteria described above are applied to the item selection after the initial item selection.

<Selection of initial items according to the second embodiment>
In the second embodiment, when measuring the second ability, the ability value of the past examinee's first ability is an explanatory variable, and the ability value of the past examinee's second ability is an objective variable. A regression equation is used. By substituting the final estimated value of the first ability of the current examinee into this simple regression equation as an explanatory variable, the initial estimated value θ ₀ of the second ability is calculated. The second adaptive test uses the initial term corresponding to this second initial estimate of ability θ ₀ .

In the second embodiment, when measuring the third ability, the ability value of the past examinee's first ability and the ability value of the second ability are the explanatory variables, and the ability value of the third ability is the objective variable. A multiple regression equation is used. By substituting the final estimated value of the first ability and the final estimated value of the second ability of the current examinee into this multiple regression equation as explanatory variables, the initial estimated value θ ₀ of the third ability is calculated. . The third adaptive test uses the initial item corresponding to the initial estimate θ ₀ of this third ability.

The measurement of the fourth and subsequent abilities is similar to the measurement of the third ability.
The above-mentioned simple regression equation and multiple regression equation are set in advance from actual measurement data of past examinees.

<Ability Estimation Method>
In the following, the ability estimation performed in the nth adaptive test for the nth ability is performed using the prior obtained based on the final estimate calculated in the adaptive test for the n-1th ability. explain what to do.

For example, in the first adaptive test, which is performed first, initial items are selected according to the first example above, and capability estimation is performed by the EAP estimation method with the prior distribution as the standard normal distribution.

If there is a correlation between abilities and the initial item is selected using any of the second through fourth examples above, then to some extent the next ability can be predicted from the already measured ability. Therefore, in the present embodiment, the initial estimated value of the adaptive test to be executed next is calculated based on the final estimated value of the previously executed adaptive test and the regression equation, and based on the initial estimated value determines the priors to be used in the next adaptive test run.

In this embodiment, using the initial estimated value θh ₀ of the ability predicted from the regression model and the estimated standard error of the prediction error of the objective variable in the regression model given by equations (13) and (14) σh , the prior distribution is denoted as N(θh ₀ , σh ² ). Using this prior distribution, capability estimation is performed by the EAP estimation method.

Here, equation (13) is an equation used in the case of a simple regression model, and equation (14) is an equation used in the case of a multiple regression model. n is the number of examinees, p is the degree of freedom of the objective variable, sse is the sum of squares of errors, ss _x is the sum of squares of the explanatory variables, xa is the mean of the explanatory variables, and _x0 is the _value of the subject's objective variable. . Di ² represents the square of the Mahalanobis distance and is defined by equation (15).

Here, ss ^jk is the (i, j) component of the inverse matrix of the variance-covariance matrix between explanatory variables.

<Comparison between the prior art and the present embodiment>
Non-Patent Literature 1 and Non-Patent Literature 2 describe a method of selecting an item for starting the next ability estimation using a regression model and a method of determining the measurement order of a plurality of abilities.

Non-Patent Document 1 describes an adaptive test for attainment testing that deals with multiple competence areas, applies a regression model to the relationship between multiple competence areas, and uses items in multiple competency areas as item selection criteria. The main method is to combine and present by In the data simulation, 232 items of tests divided into 12 abilities were conducted on paper for 365 Navy fire protection engineers, and the amount of information in the test and the length of the test time were examined. compares adaptive tests with conventional tests. As a result of this simulation, it was not sufficient to guarantee the application of the regression model to the relationship between multiple abilities against the correlation between multiple test takers, and the number of items in each of the 12 ability tests was different. However, there was no significant difference in the amount of information between the two abilities. Among the items, there are items for which the measurement accuracy is not improved. This simulation revealed that there was no need to use such low-precision terms. On average, the adaptive test used about half as many items as the conventional test, with the smallest being 18% and the largest being 80%. However, the information curve obtained from the adaptive test was not intrinsically meaningful with respect to the information curve obtained from the conventional test. That is, the method of Non-Patent Document 1 contributed to a significant reduction in the test time compared to the conventional method, but because the accuracy of measurement of ability decreases, the method of ability estimation in the initial stage of adaptive testing And there were issues with item selection criteria.

Non-Patent Document 2 compares adaptive testing with conventional testing and explains its advantages. Adaptive tests reduce test time when measuring changes in individual test-taker behavior or student achievement, comprehension, and performance as a result of education in a variety of settings, such as counseling or education. A specific example is ASRT (Adaptive Self-Referenced Testing). The ASRT measures 12 skill areas, such as reading and mathematics, and asks students to measure their areas of interest in these areas with a series of test items. The first test is conducted as a general adaptive test, and when the same students are given this test again at a later date, the items presented in the previous test are excluded from the measurement. The initial performance values for the second and subsequent tests use the estimated values from the previous test, and the measurement ends when there is a significant change from this estimated value, or when the estimated value is fixed. By doing this process each time, it is possible to track the status of changes in the student.

Non-Patent Documents 3 to 5 describe multi-dimensional adaptive tests that streamline ability estimation.

A multidimensional adaptive test uses the fact that there is a correlation between each ability to be measured, and not only the ability corresponding to the presented item from the response each time one item is presented, but also the multidimensional adaptive test is measured. Measure all abilities.

In this way, in the multidimensional adaptive test, since all abilities are estimated for each item, the amount of calculation increases and the load on the server that performs the processing increases. The increase in computational complexity becomes more pronounced as the number of capabilities to be measured increases.

In addition, in the multidimensional adaptive test, it is difficult for the examinee to recognize which ability the next item is to measure, which may confuse the examinee. Multidimensional adaptive tests are not suitable for use when random presentation of items for each ability is not appropriate, such as when you want to present examples in advance to measure each ability.

In the ability measurement process according to this embodiment, when a plurality of abilities are measured by the adaptive test, lengthening of the test time is suppressed. In the ability measurement process according to the present embodiment, it is possible to reduce the number of items presented to the examinee when there is a strong correlation between a plurality of abilities.

In the ability measurement process according to the present embodiment, in the first adaptive test, the adaptive test is started by assuming, for example, an average ability value of zero as an initial estimated value of ability. When measuring the second and subsequent abilities, an initial estimate is calculated using the regression equation obtained from the data based on the correlation between abilities and the results of the adaptive tests that have been performed, and the initial estimate is used. selects the initial item for the next adaptive test to be run.

Adaptive testing uses, for example, the standard normal distribution as a prior distribution for estimating ability. In the ability measurement process according to the present embodiment, the prior distribution with the initial estimated value of ability as the average and the standard deviation of the prediction error of the objective variable in the regression equation for obtaining the initial estimated value is used as the prior distribution when estimating ability. used for

In the ability measurement process according to the present embodiment as described above, it is possible to perform a plurality of ability measurements efficiently and accurately while solving the problems of Non-Patent Documents 1 to 5 described above. In the ability measurement process according to the present embodiment, since the relationship between each ability to be measured and its item is clear, it is possible to perform measurement that is easy for the examinee to understand, such as by presenting an example before the ability measurement. .

<Summary>
In the present embodiment described above, when a plurality of abilities are measured using a plurality of adaptive tests, the fact that each ability has a strong correlation is used to reduce the number of items presented to the examinee. can improve the efficiency of adaptive testing.

In this embodiment, after the final estimated value of a certain ability is calculated, when measuring the next ability, the initial estimated value is calculated based on the final estimated value measured before this next ability. . For subsequent performance measurements, initial items are selected using the computed initial estimates, prior distributions are determined, and performance estimates are computed using Bayesian performance estimation. Regarding the selection of the initial items, the first to fourth examples were explained, but the result of the simulation was that the fourth example was the most effective.

As the prior distribution used for estimating ability, a normal distribution can be used in which the mean is the initial estimate of the ability and the standard deviation is the estimated variance of the prediction error of the objective variable in the regression model.
Regarding the item selection criteria, it is preferable to use MFI when comprehensively considering the load on the server and the degree of reduction in the number of items.

When the ability measurement process according to the present embodiment is executed under the above conditions, the number of tests that conventionally required about 26.5 items per person on average can be reduced to about 20 to 22 items on average. .

In this embodiment, since the test time is shortened, it is possible to reduce factors that hinder the measurement, such as examinee's fatigue and reduced concentration, so that the examinee can fully exert his/her ability. can.

The test operator can reduce the administrative burden of maintaining, updating, and expanding the item pool because the exposure of item content is reduced when the number of selected items is reduced. In addition, since multiple abilities can be measured in a limited amount of time, opportunities for introducing adaptive tests can be increased. In addition, the test operator can increase the number of test executions by shortening the test time, increase the turnover rate of the equipment, and increase profits.

A multidimensional adaptive test is characterized in that the abilities and fields related to the items presented to the examinee are not fixed, but in this embodiment, the abilities and fields of the items presented to the examinee can be divided. Therefore, it is possible to conduct a test after giving explanations, presenting example questions, or practicing for each ability.

It should be noted that the present invention is not limited to the embodiments described above, and each embodiment can be implemented by deleting, adding, or changing constituent elements. Also, by appropriately combining or exchanging each component of each embodiment, it is possible to implement it in a further different form. In this way, even if the embodiments are directly different from the embodiments described above, those having the same meaning as the present invention are described as the embodiments of the present invention, and the description thereof is omitted. there is

DESCRIPTION OF SYMBOLS 1...Ability measuring device 2...Input device 3...Output device 4...Storage device 5...Processing device 6...Bus 7...Program 8...Item pool 9...Answer data 10...Estimated data 111 ... first ability measurement data, 112 ... second ability measurement data, 113 ... third ability measurement data, 121 ... first regression model, 122 ... second regression model, 13a, 13b ... initial item data, 14a, 14b prior distribution data 15 control unit 161 first ability measurement unit 162 second ability measurement unit 163 third ability measurement unit 17 initial item selection unit 18 prior Distribution determination part.

Claims (7)

a storage device for storing item data including a plurality of items used for measuring a plurality of abilities;
a processing device that executes an adaptive test for each of the plurality of abilities using the item data;
and
The plurality of abilities includes a first ability, a second ability measured after the first ability, and a third ability measured after the second ability,
The processing device is
selecting an initial item for a second adaptive test for the second ability based on a first final estimate of the first adaptive test for the first ability; determine the prior distribution used in ability estimation,
selecting an initial item of a third adaptive test for the third ability based on a second final estimate of the second adaptive test to be used in the ability estimation of the third adaptive test determine the prior distribution ,
The processing device is
calculating an initial estimate of the second adaptive test based on the first final estimate and the first regression equation; and calculating an initial estimate of the second adaptive test based on the initial estimate of the second adaptive test; Select the initial item of the adaptive test of 2,
calculating an initial estimate of the third adaptive test based on the second final estimate and a second regression equation; and calculating an initial estimate of the third adaptive test based on the initial estimate of the third adaptive test; Select the initial item of the adaptive test of 3,
The processing device is
The ability estimation of the second adaptive test is a prior distribution with the initial estimated value of the second adaptive test as the mean value and the standard error of the prediction error of the objective variable in the first regression equation as the standard deviation. Let the prior distribution used in
The ability estimation of the third adaptive test is a prior distribution with the initial estimated value of the third adaptive test as the mean value and the standard error of the prediction error of the objective variable in the second regression equation as the standard deviation. Let the prior distribution used in
Ability measuring device. The processing device is
Using a Bayesian method in the ability estimation of the second adaptive test and the ability estimation of the third adaptive test,
selecting the item with the maximum Fisher information amount for the initial estimate of the second adaptive test as the initial item of the second adaptive test;
Selecting the item with the maximum Fisher information for the initial estimate of the third adaptive test as the initial item of the third adaptive test;
The ability measuring device according to claim 1 . a storage device for storing item data including a plurality of items used for measuring a plurality of abilities;
a processing device that executes an adaptive test for each of the plurality of abilities using the item data;
and
The plurality of abilities includes a first ability, a second ability measured after the first ability, and a third ability measured after the second ability,
The processing device is
selecting an initial item for a second adaptive test for the second ability based on a first final estimate of the first adaptive test for the first ability; determine the prior distribution used in ability estimation,
selecting an initial item for a third adaptive test for the third ability based on the first final estimate and a second final estimate of the second adaptive test; determine the prior distribution used in proficiency estimation for the type test;
The processing device is
calculating an initial estimate of the second adaptive test based on the first final estimate and the first regression equation; and calculating an initial estimate of the second adaptive test based on the initial estimate of the second adaptive test; Select the initial item of the adaptive test of 2,
calculating an initial estimate of the third adaptive test based on the first final estimate, the second final estimate, and a second regression equation; and calculating an initial estimate of the third adaptive test selecting an initial item for the third adaptive test based on the estimate;
The processing device is
The ability estimation of the second adaptive test is a prior distribution with the initial estimated value of the second adaptive test as the mean value and the standard error of the prediction error of the objective variable in the first regression equation as the standard deviation. Let the prior distribution used in
The ability estimation of the third adaptive test is a prior distribution with the initial estimated value of the third adaptive test as the mean value and the standard error of the prediction error of the objective variable in the second regression equation as the standard deviation. Let the prior distribution used in
Ability measuring device. The processing device is
Using a Bayesian method in the ability estimation of the second adaptive test and the ability estimation of the third adaptive test,
selecting the item with the maximum Fisher information amount for the initial estimate of the second adaptive test as the initial item of the second adaptive test;
Selecting the item with the maximum Fisher information for the initial estimate of the third adaptive test as the initial item of the third adaptive test;
4. The ability measuring device according to claim 3. 4. The second regression equation comprises a multiple regression equation for selecting initial items for the third adaptive test based on the first final estimate and the second final estimate. 3 ability measuring device. to the computer,
a function of executing a first adaptive test that selects items used to measure the first ability from the item data stored in the storage device;
An initial estimate of a second adaptive test for a second ability measured after the first ability based on a first final estimate of the first adaptive test and a first regression equation. and selecting an initial item for the second adaptive test based on the initial estimate for the second adaptive test, taking the initial estimate for the second adaptive test as an average value, and A function of determining a distribution whose standard deviation is the standard error of the prediction error of the objective variable in the first regression equation as the prior distribution used in the ability estimation of the second adaptive test;
Based on the initial item of the second adaptive test, the prior distribution used in ability estimation of the second adaptive test, and the item data stored in the storage device, the second the ability to run adaptive tests for
An initial estimate of a third adaptive test for a third ability measured after the second ability based on a second final estimate of the second adaptive test and a second regression equation. and selecting an initial item for the third adaptive test based on the initial estimate for the third adaptive test, taking the initial estimate for the third adaptive test as an average value, and A function of determining a distribution whose standard deviation is the standard error of the prediction error of the objective variable in the second regression equation as the prior distribution used in the ability estimation of the third adaptive test;
Based on the initial items of the third adaptive test, the prior distribution used in ability estimation of the third adaptive test, and the item data stored in the storage device, the third the ability to run adaptive tests for
program to make it happen. executing, by a computer, a first adaptive test that selects items used to measure the first ability from item data stored in a storage device;
measured by the computer after the first ability based on the first final estimate of the first adaptive test , the first regression equation , and the item data stored in the storage device calculating an initial estimate of a second adaptive test for a second ability in which the A distribution with an initial estimated value of the second adaptive test as the mean value and a standard deviation as the standard error of the prediction error of the objective variable in the first regression equation is used in estimating the ability of the second adaptive test. determining as a prior distribution;
by the computer, based on the initial items of the second adaptive test, the prior distribution used in estimating ability of the second adaptive test, and the item data stored in the storage device; , performing the second adaptive test;
measured by the computer after the second ability based on a second final estimate of the second adaptive test , a second regression equation, and the item data stored in the storage device calculating an initial estimate of a third adaptive test for a third ability that A distribution with an initial estimated value of the third adaptive test as the mean value and a standard deviation as the standard error of the prediction error of the objective variable in the second regression equation is used in the ability estimation of the third adaptive test. determining as a prior distribution;
By the computer, based on the initial items of the third adaptive test, the prior distribution used in estimating ability of the third adaptive test, and the item data stored in the storage device , performing said third adaptive test;
Ability measurement method comprising

Images