Chunked Fread — fread_chunked • barketools

Usage

fread_chunked(file, chunk_size, callback, header = FALSE, verbose = TRUE, ...)

file: File name in working directory, path to file (passed through base::path.expand for convenience), or a URL starting http://, file://, etc. Compressed files with extension ‘.gz’ and ‘.bz2’ are supported if the R.utils package is installed.
chunk_size: Number of rows that is read per chunk.
callback: A function taking two arguments x (the data of the current chunk) and chunk (the zero-based number of the current chunk). Use this function to process the chunked data, e.g., by transforming it and writing it to a database.
header: Argument passed to data.table::fread. Currently, only FALSE is supported, so that it is suggested to supply col.names as well.
verbose: If TRUE, print progress information.
...: Arguments passed to data.table::fread.