Package 'curtail'

Title: A package for test curtailment
Description: A package for creating and assessing the performance of curtailed tests. A curtailed test is a variable-length test, which allows for early stopping of item administration when further items are unlikely or unable to change the final (classification) decision. The package allows for creating and assessing deterministically and stochastically (based on empirical proportions) curtailed tests. In future versions of the package, model-based stochastic curtailment may be added. The package allows for test curtailment through the creation of look-up tables, providing item-specific cut-off values for early stopping of item administration. In addition, the package allows for assessing efficiency (number of items administered to arrive at a final decision) and accuracy (concordance between the final decision based on curtailed and full-length test).
Authors: Marjolein Fokkema [aut, cre]
Maintainer: Marjolein Fokkema <[email protected]>
License: GPL-2
Version: 0.2
Built: 2026-05-27 08:51:47 UTC
Source: https://github.com/marjoleinF/curtail

Help Index

Simulate deterministic curtailment on item scores of a test

Description

Simulates deterministic curtailment on an existing dataset of item scores, based on a user-specified cut-off value.

Usage

Curtail(dataset.test, Xstar, highest = NULL, lowest = NULL, plot = TRUE)

Arguments

dataset.test

A dataframe containing item scores only.
Xstar

Cut-off value to be used for classifying observations as 'at risk' (test-score values greater than or equal to the cut-off value) or 'not at risk' (test-score values less than cut-off value)
highest

Highest possible item score. If not provided, function will take the max of all observed item scores.
lowest

Lowest possible item score. If not provided, function will take the min of all observed item scores.
plot

Should a histogram of test lengths be plotted?

Details

Note that currently, only items with the same number of response categories, and the same ordering of the response categories, are supported.

The code is still under development and might change in future versions.

Value

The function prints accuracy estimates to the command line, and plots the curtailed test length distribution. In addition, the function invisibly returns a list with the following elements:

test.results

data.frame with columns full.lenght.decision (classification decsion according to full-length test); curtailed.decision (classification decision according to curtailed test administration); current.item (item at which testing was halted); current.score (cumulative testscore at item at which testing was halted).
curtailed.test.length.distribution

Descriptive statistics of number of items administered and number of tests curtailed.
confusion.martrix

Confusion matrix of full-length and curtailed test classification decisions.
accuracy

Correct classification rate (accuracy), sensitivity and specificity.

References

Fokkema, M., Smits, N., Finkelman, M. D., Kelderman, H., & Cuijpers, P. (2014). Curtailment: A method to reduce the length of self-report questionnaires while maintaining diagnostic accuracy. Psychiatry Research 215, 477-482.

Fokkema, M., Smits, N., Kelderman, H., Carlier, I.V. & Van Hemert, A.M. (2014). Combining decision trees and stochastic curtailment for assessment length reduction of test batteries used for classification. Applied Psychological Measurement, 38(1), 3-17.

Finkelman, M.D., Smits, N., Kim, W. & Riley, B. (2012). Curtailment and stochastic curtailment to shorten the CES-D. Applied Psychological Measurement, 36(8), 632-658.

Examples

tmp <- Curtail(itemscores, 19)
  tmp$curtailed.test.length.distribution

Dataset for illustrating curtailment functions

Description

Dataset containing 1000 simulated responses to 20 items on a mental-health questionnaire. Items are scored 0, 1, 2 or 3. The dataset is used in the examples to illustrate the functions. Observations are classified as 'at risk' with a total score after admistration of item 20, equal to or greater than 19 (i.e., a cut-off score of 19 is used).

Examples

summary(itemscores)

Simulates stochastic curtailment on item scores of a test

Description

Simulates stochastic curtailment on an existing test dataset of item scores, based on an existing training dataset of item scores, and a user-specified cut-off value.

Usage

stochCurtail(dataset.train, dataset.test = NULL, Xstar, gamma0=.95, 
  gamma1=.95, plot = TRUE)

Arguments

dataset.train

A dataframe, containing item scores only, which will be used to derive the probabilities of obtaining a final test score greater than, or equal to, the cut-off value, based on the current cumulative score
dataset.test

A dataframe containing item scores only. Curtailment will be simulated on these observations. When no test dataset is specified, curtailment will be simulated on the training dataset.
Xstar

Cut-off value to be used for classifying observations as 'at risk' (test-score values greater than or equal to the cut-off value) or 'not at risk' (test-score values less than cut-off value)
gamma0

The threshold for the probability, calculated using the 'not-at-risk' training observations, that the classification decision based on the stochastically curtailed version will match that of the full-length instrument.
gamma1

The threshold for the probability, calculated using the 'at-risk' training observations, that the classification decision based on the stochastically curtailed version will match that of the full-length instrument.
plot

Should a histogram of test lengths be plotted?

Details

The code is still under development and might change in future versions.

Value

The function prints accuracy estimates to the command line, and plots the curtailed test length distribution. In addition, the function invisibly returns a list with the following elements:

test.results

data.frame with columns full.lenght.decision (classification decsion according to full-length test); curtailed.decision (classification decision according to curtailed test administration); current.item (item at which testing was halted); current.score (cumulative testscore at item at which testing was halted).
curtailed.test.length.distribution

Descriptive statistics of number of items administered and number of tests curtailed.
confusion.martrix

Confusion matrix of full-length and curtailed test classification decisions.
accuracy

Correct classification rate (accuracy), sensitivity and specificity.

References

Fokkema, M., Smits, N., Finkelman, M. D., Kelderman, H., & Cuijpers, P. (2014). Curtailment: A method to reduce the length of self-report questionnaires while maintaining diagnostic accuracy. Psychiatry Research 215, 477-482.

Fokkema, M., Smits, N., Kelderman, H., Carlier, I.V. & Van Hemert, A.M. (2014). Combining decision trees and stochastic curtailment for assessment length reduction of test batteries used for classification. Applied Psychological Measurement, 38(1), 3-17.

Finkelman, M.D., Smits, N., Kim, W. & Riley, B. (2012). Curtailment and stochastic curtailment to shorten the CES-D. Applied Psychological Measurement, 36(8), 632-658.

Examples

## obtain a test and training dataset
  set.seed(32061983)
  samp <- sample(1:1000, 500); train <- samp[1:500]
  trainingdata <- itemscores[train,]
  testdata <- itemscores[-train,]
  tmp1 <- stochCurtail(trainingdata, testdata, 19)
  tmp1$curtailed.test.length.distribution
  ## try lower gamma values for earlier stopping, but lower accuracy:
  tmp2 <- stochCurtail(trainingdata, testdata, 19, gamma0=.75, gamma1=.75)
  tmp2$curtailed.test.length.distribution

Simulate stochastic curtailment using leave-one-out CV

Description

Simulates stochastic curtailment on an existing dataset of item scores, using leave-one-out (LOO) cross validation, and a user-specified cut-off value. For a given observation, all other observations will be used to derive the probabilities of obtaining a final test score greater than or equal to the cut-off value.

Usage

stochCurtailXval(dataset, Xstar, gamma0 = .95, gamma1 = .95, plot = TRUE, 
  verbose = FALSE)

Arguments

dataset

A dataframe, containing item scores only.
Xstar

Cut-off value to be used for classifying observations as 'at risk' (test-score values greater than or equal to the cut-off value) or 'not at risk' (test-score values less than cut-off value)
gamma0

The threshold for the probability, calculated using the 'not-at-risk' training observations, that the classification decision based on the stochastically curtailed version will match that of the full-length instrument.
gamma1

The threshold for the probability, calculated using the 'at-risk' training observations, that the classification decision based on the stochastically curtailed version will match that of the full-length instrument.
plot

Should test length distribution be plotted?
verbose

Should progress be plotted to the command line?

Details

The code is still under development and might change in future versions.

Value

The function prints accuracy estimates to the command line, and plots the curtailed test length distribution. In addition, the function invisibly returns a list with the following elements:

test.results

data.frame with columns full.lenght.decision (classification decsion according to full-length test); curtailed.decision (classification decision according to curtailed test administration); current.item (item at which testing was halted); current.score (cumulative testscore at item at which testing was halted).
curtailed.test.length.distribution

Descriptive statistics of number of items administered and number of tests curtailed.
confusion.martrix

Confusion matrix of full-length and curtailed test classification decisions.
accuracy

Correct classification rate (accuracy), sensitivity and specificity.

References

Fokkema, M., Smits, N., Finkelman, M. D., Kelderman, H., & Cuijpers, P. (2014). Curtailment: A method to reduce the length of self-report questionnaires while maintaining diagnostic accuracy. Psychiatry Research 215, 477-482.

Fokkema, M., Smits, N., Kelderman, H., Carlier, I.V. & Van Hemert, A.M. (2014). Combining decision trees and stochastic curtailment for assessment length reduction of test batteries used for classification. Applied Psychological Measurement, 38(1), 3-17.

Finkelman, M.D., Smits, N., Kim, W. & Riley, B. (2012). Curtailment and stochastic curtailment to shorten the CES-D. Applied Psychological Measurement, 36(8), 632-658.

Examples

## Only first 100 observations are specified here, to reduce computation time: 
    tmp <- stochCurtailXval(itemscores[1:100,], 19, gamma0=.95, gamma1=.95) 
    tmp$curtailed.test.length.distribution

Create a look-up table for stochastic curtailment

Description

Creates a look-up table for early stopping of item administration, for stochastic curtailment.

Usage

stochTable(dataset.train, Xstar, gamma0 = 0.95, gamma1 = 0.95)

Arguments

dataset.train

Dataframe, containing item scores only
Xstar

cut-off value for the final test score to be used
gamma0

The threshold for the probability, calculated using the 'not-at-risk' training observations, that the classification decision based on the stochastically curtailed version will match that of the full-length instrument.
gamma1

The threshold for the probability, calculated using the 'at-risk' training observations, that the classification decision based on the stochastically curtailed version will match that of the full-length instrument.

Details

The code is still under development and might change in future versions.

Value

Returns a look-up table with cut-off values for every item, to be used for early stopping of item administration.

References

Fokkema, M., Smits, N., Finkelman, M. D., Kelderman, H., & Cuijpers, P. (2014). Curtailment: A method to reduce the length of self-report questionnaires while maintaining diagnostic accuracy. Psychiatry Research 215, 477-482.

Fokkema, M., Smits, N., Kelderman, H., Carlier, I.V. & Van Hemert, A.M. (2014). Combining decision trees and stochastic curtailment for assessment length reduction of test batteries used for classification. Applied Psychological Measurement, 38(1), 3-17.

Finkelman, M.D., Smits, N., Kim, W. & Riley, B. (2012). Curtailment and stochastic curtailment to shorten the CES-D. Applied Psychological Measurement, 36(8), 632-658.

Examples

stochTable(itemscores, 19)
  # try lower gamma values for earlier stopping:
  stochTable(itemscores, 19, gamma0 = .75, gamma1 = .75)

Create a look-up table for deterministic curtailment

Description

Creates a look-up table for early stopping of item administration, for deterministic curtailment.

Usage

Table(dataset.train = NULL, Xstar, nitems = NULL, highest = NULL, lowest  = NULL)

Arguments

dataset.train

Optional, a dataframe, containing item scores only
Xstar

cut-off value for the final test score to be used
nitems

number of items in questionnaire (assumed equal for all items)
highest

Optional, highest theoretically possible item score (assumed equal for all items)
lowest

Optional, lowest theoretically possible item score (assumed equal for all items)

Details

Note that either dataset.train needs to be specified, or nitems, highest and lowest.

The code is still under development and may change in future versions.

Value

Returns a look-up table with cut-off values for every item, to be used for early stopping of item administration.

References

Fokkema, M., Smits, N., Finkelman, M. D., Kelderman, H., & Cuijpers, P. (2014). Curtailment: A method to reduce the length of self-report questionnaires while maintaining diagnostic accuracy. Psychiatry Research 215, 477-482.

Fokkema, M., Smits, N., Kelderman, H., Carlier, I.V. & Van Hemert, A.M. (2014). Combining decision trees and stochastic curtailment for assessment length reduction of test batteries used for classification. Applied Psychological Measurement, 38(1), 3-17.

Finkelman, M.D., Smits, N., Kim, W. & Riley, B. (2012). Curtailment and stochastic curtailment to shorten the CES-D. Applied Psychological Measurement, 36(8), 632-658.

Examples

Table(itemscores, 19)
  Table(Xstar = 19, nitems = 20, highest = 3, lowest = 0)