Skip to content

fst

Lightning Fast Serialization of Data Frames

v0.9.8 · Feb 7, 2022 · AGPL-3 | file LICENSE
CRAN GitHub Mirror

Description

Multithreaded serialization of compressed data frames using the 'fst' format. The 'fst' format allows for full random access of stored data and a wide range of compression settings using the LZ4 and ZSTD compressors.

Downloads

14.3K

Last 30 days

974th

14.3K

Last 90 days

14.3K

Last year

CRAN Check Status

1 ERROR
13 OK
Show all 14 flavors
Flavor Status
r-devel-linux-x86_64-debian-clang OK
r-devel-linux-x86_64-debian-gcc OK
r-devel-linux-x86_64-fedora-clang ERROR
r-devel-linux-x86_64-fedora-gcc OK
r-devel-macos-arm64 OK
r-devel-windows-x86_64 OK
r-oldrel-macos-arm64 OK
r-oldrel-macos-x86_64 OK
r-oldrel-windows-x86_64 OK
r-patched-linux-x86_64 OK
r-release-linux-x86_64 OK
r-release-macos-arm64 OK
r-release-macos-x86_64 OK
r-release-windows-x86_64 OK
Check details (14 non-OK)
OK r-devel-linux-x86_64-debian-clang

*


            

            

              

                OK
                r-devel-linux-x86_64-debian-gcc
              

              
*

              

            

            

              

                ERROR
                r-devel-linux-x86_64-fedora-clang
              

              
tests

              
  Running ‘testthat.R’ [52s/60s]
Running the tests in ‘tests/testthat.R’ failed.
Complete output:
  > 
  > # required packages
  > library(testthat)
  > library(fst)
  > 
  > # run tests
  > test_check("fst")
  OMP: Warning #96: Cannot form a team with 24 threads, using 2 instead.
  OMP: Hint Consider unsetting KMP_DEVICE_THREAD_LIMIT (KMP_ALL_THREADS), KMP_TEAMS_THREAD_LIMIT, and OMP_THREAD_LIMIT (if any are set).
  
   *** caught segfault ***
  address 0x1f1c00001f2b, cause 'memory not mapped'
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  *** recursive gc invocation
  
  Traceback:
   1: make.unique(as.character(rows))
   2: `[.data.frame`(difftable, uneq, )
   3: difftable[uneq, ]
   4: paste(difftable[uneq, ][1:min(25, sum(uneq, na.rm = TRUE)), 1],     collapse = ",")
   5: paste("args: col:", col, "| from:", from, "| to:", to, "| setColumns:",     sel_columns, "| compress:", compress, "| tot_length", tot_length,     " cols sub_dt:", ncol(sub_dt), ", rows sub_dt:", nrow(sub_dt),     "cols data:", ncol(data), ", rows data:", nrow(data), "head sub_dt:",     paste(sub_dt[1:10, 1], collapse = ","), "head data:", paste(data[1:10,         1], collapse = ","), "unequals:", sum(uneq), "uneq rows sub_dt1",     paste(difftable[uneq, ][1:min(25, sum(uneq, na.rm = TRUE)),         1], collapse = ","), "uneq rows sub_dt2", paste(difftable[uneq,         ][1:min(25, sum(uneq, na.rm = TRUE)), 2], collapse = ","),     "uneq rows sub_dt3", paste(difftable[uneq, ][1:min(25, sum(uneq,         na.rm = TRUE)), 3], collapse = ","))
   6: test_write_read(col, 1L + block_start * block_size, last_row,     NULL, compression)
   7: blocktests(type, 0, 1, 80L)
   8: blocktestsingletype("Ylog")
   9: eval(code, test_env)
  10: eval(code, test_env)
  11: withCallingHandlers({    eval(code, test_env)    new_expectations <- the$test_expectations > starting_expectations    if (snapshot_skipped) {        skip("On CRAN")    }    else if (!new_expectations && skip_on_empty) {        skip_empty()    }}, expectation = handle_expectation, packageNotFoundError = function(e) {    if (on_cran()) {        skip(paste0("{", e$package, "} is not installed."))    }}, snapshot_on_cran = function(cnd) {    snapshot_skipped <<- TRUE    invokeRestart("muffle_cran_snapshot")}, skip = handle_skip, warning = handle_warning, message = handle_message,     error = handle_error, interrupt = handle_interrupt)
  12: doTryCatch(return(expr), name, parentenv, handler)
  13: tryCatchOne(expr, names, parentenv, handlers[[1L]])
  14: tryCatchList(expr, classes, parentenv, handlers)
  15: tryCatch(withCallingHandlers({    eval(code, test_env)    new_expectations <- the$test_expectations > starting_expectations    if (snapshot_skipped) {        skip("On CRAN")    }    else if (!new_expectations && skip_on_empty) {        skip_empty()    }}, expectation = handle_expectation, packageNotFoundError = function(e) {    if (on_cran()) {        skip(paste0("{", e$package, "} is not installed."))    }}, snapshot_on_cran = function(cnd) {    snapshot_skipped <<- TRUE    invokeRestart("muffle_cran_snapshot")}, skip = handle_skip, warning = handle_warning, message = handle_message,     error = handle_error, interrupt = handle_interrupt), error = handle_fatal)
  16: doWithOneRestart(return(expr), restart)
  17: withOneRestart(expr, restarts[[1L]])
  18: withRestarts(tryCatch(withCallingHandlers({    eval(code, test_env)    new_expectations <- the$test_expectations > starting_expectations    if (snapshot_skipped) {        skip("On CRAN")    }    else if (!new_expectations && skip_on_empty) {        skip_empty()    }}, expectation = handle_expectation, packageNotFoundError = function(e) {    if (on_cran()) {        skip(paste0("{", e$package, "} is not installed."))    }}, snapshot_on_cran = function(cnd) {    snapshot_skipped <<- TRUE    invokeRestart("muffle_cran_snapshot")}, skip = handle_skip, warning = handle_warning, message = handle_message,     error = handle_error, interrupt = handle_interrupt), error = handle_fatal),     end_test = function() {    })
  19: test_code(code, parent.frame())
  20: test_that("Logical column block tests", {    blocktestsingletype("Ylog")})
  21: eval(code, test_env)
  22: eval(code, test_env)
  23: withCallingHandlers({    eval(code, test_env)    new_expectations <- the$test_expectations > starting_expectations    if (snapshot_skipped) {        skip("On CRAN")    }    else if (!new_expectations && skip_on_empty) {        skip_empty()    }}, expectation = handle_expectation, packageNotFoundError = function(e) {    if (on_cran()) {        skip(paste0("{", e$package, "} is not installed."))    }}, snapshot_on_cran = function(cnd) {    snapshot_skipped <<- TRUE    invokeRestart("muffle_cran_snapshot")}, skip = handle_skip, warning = handle_warning, message = handle_message,     error = handle_error, interrupt = handle_interrupt)
  24: doTryCatch(return(expr), name, parentenv, handler)
  25: tryCatchOne(expr, names, parentenv, handlers[[1L]])
  26: tryCatchList(expr, classes, parentenv, handlers)
  27: tryCatch(withCallingHandlers({    eval(code, test_env)    new_expectations <- the$test_expectations > starting_expectations    if (snapshot_skipped) {        skip("On CRAN")    }    else if (!new_expectations && skip_on_empty) {        skip_empty()    }}, expectation = handle_expectation, packageNotFoundError = function(e) {    if (on_cran()) {        skip(paste0("{", e$package, "} is not installed."))    }}, snapshot_on_cran = function(cnd) {    snapshot_skipped <<- TRUE    invokeRestart("muffle_cran_snapshot")}, skip = handle_skip, warning = handle_warning, message = handle_message,     error = handle_error, interrupt = handle_interrupt), error = handle_fatal)
  28: doWithOneRestart(return(expr), restart)
  29: withOneRestart(expr, restarts[[1L]])
  30: withRestarts(tryCatch(withCallingHandlers({    eval(code, test_env)    new_expectations <- the$test_expectations > starting_expectations    if (snapshot_skipped) {        skip("On CRAN")    }    else if (!new_expectations && skip_on_empty) {        skip_empty()    }}, expectation = handle_expectation, packageNotFoundError = function(e) {    if (on_cran()) {        skip(paste0("{", e$package, "} is not installed."))    }}, snapshot_on_cran = function(cnd) {    snapshot_skipped <<- TRUE    invokeRestart("muffle_cran_snapshot")}, skip = handle_skip, warning = handle_warning, message = handle_message,     error = handle_error, interrupt = handle_interrupt), error = handle_fatal),     end_test = function() {    })
  31: test_code(code = exprs, env = env, reporter = get_reporter() %||%     StopReporter$new())
  32: source_file(path, env = env(env), desc = desc, shuffle = shuffle,     error_call = error_call)
  33: FUN(X[[i]], ...)
  34: lapply(test_paths, test_one_file, env = env, desc = desc, shuffle = shuffle,     error_call = error_call)
  35: doTryCatch(return(expr), name, parentenv, handler)
  36: tryCatchOne(expr, names, parentenv, handlers[[1L]])
  37: tryCatchList(expr, classes, parentenv, handlers)
  38: tryCatch(code, testthat_abort_reporter = function(cnd) {    cat(conditionMessage(cnd), "\n")    NULL})
  39: with_reporter(reporters$multi, lapply(test_paths, test_one_file,     env = env, desc = desc, shuffle = shuffle, error_call = error_call))
  40: test_files_serial(test_dir = test_dir, test_package = test_package,     test_paths = test_paths, load_helpers = load_helpers, reporter = reporter,     env = env, stop_on_failure = stop_on_failure, stop_on_warning = stop_on_warning,     desc = desc, load_package = load_package, shuffle = shuffle,     error_call = error_call)
  41: test_files(test_dir = path, test_paths = test_paths, test_package = package,     reporter = reporter, load_helpers = load_helpers, env = env,     stop_on_failure = stop_on_failure, stop_on_warning = stop_on_warning,     load_package = load_package, parallel = parallel, shuffle = shuffle)
  42: test_dir("testthat", package = package, reporter = reporter,     ..., load_package = "installed")
  43: test_check("fst")
  An irrecoverable exception occurred. R is aborting now ...

            

            

              

                OK
                r-devel-linux-x86_64-fedora-gcc
              

              
*

              

            

            

              

                OK
                r-devel-macos-arm64
              

              
*

              

            

            

              

                OK
                r-devel-windows-x86_64
              

              
*

              

            

            

              

                OK
                r-oldrel-macos-arm64
              

              
*

              

            

            

              

                OK
                r-oldrel-macos-x86_64
              

              
*

              

            

            

              

                OK
                r-oldrel-windows-x86_64
              

              
*

              

            

            

              

                OK
                r-patched-linux-x86_64
              

              
*

              

            

            

              

                OK
                r-release-linux-x86_64
              

              
*

              

            

            

              

                OK
                r-release-macos-arm64
              

              
*

              

            

            

              

                OK
                r-release-macos-x86_64
              

              
*

              

            

            

              

                OK
                r-release-windows-x86_64
              

              
*

              

            

          

        

      



      

      
      

        
Check History

        

          

            
              ERROR
              
                13 OK ·                 0 NOTE ·                 0 WARNING ·                 1 ERROR ·                 0 FAILURE              
              Mar 9, 2026
            
            

              

                

                  ERROR
                  r-devel-linux-x86_64-fedora-clang
                

                
tests

                
  Running ‘testthat.R’ [52s/60s]
Running the tests in ‘tests/testthat.R’ failed.
Complete output:
  > 
  > # required packages
  > library(testthat)
  > library(fst)
  > 
  > # run tests
  > test_check("fst")
  OMP: Warning #96: Cannot form a team with 24 threads, using 2 instead.
  OMP: Hint Consider unsetting KMP_DEVICE_THREAD_LIMIT (KMP_ALL_THREADS), KMP_TEAMS_THREAD_LIMIT, and OMP_THREAD_LIMIT (if any are set).
  
   *** caught segfault ***
  address 0x1f1c00001f2b, cause 'memory not mapped'

              

            

          

        

      


      
      

        
Reverse Dependencies (37)

        

          
imports

          

            CongressData
            DisImpact
            JWileymisc
            NMdata
            NMsim
            genekitr
            geneset
            hdd
            healthyAddress
            hkdatasets
            ieegio
            jagstargets
            mrgsim.parallel
            nhdplusTools
            parquetize
            prt
            qtl2fst
            qtl2pattern
            ravepipeline
            readabs
            ricu
            standartox
            tidyfst
            tidyft
          

        

        

          
suggests

          

            brms.mmrm
            drake
            expss
            grattan
            heims
            hutils
            magicaxis
            randomForestSRC
            rio
            syncdr
            targets
            weatherjoin
            wpa
          

        

      


      
      

        
Dependency Network

        

          
            

            
            
            Dependencies
            
            Reverse dependencies

            
            
            

            
            
            
            
            
            
            
            
            
            
            
            
            
            
            
            

            
            
              
              fstcore
            
            
              
              Rcpp
            

            
            
              
              CongressData
            
            
              
              DisImpact
            
            
              
              JWileymisc
            
            
              
              NMdata
            
            
              
              NMsim
            
            
              
              genekitr
            
            
              
              geneset
            
            
              
              hdd
            
            
              
              healthyAddress
            
            
              
              hkdatasets
            
            
              
              ieegio
            
            
              
              jagstargets
            
            
              
              mrgsim.parallel
            
            
              
              nhdplusTools
            
            
              
              parquetize
            
            +22 more reverse deps

            
            
            fst
          
        

      


      
      

        
Version History

        

          

            new
            0.9.8
            Mar 9, 2026