p3-shuffle

Scramble the Records in a File

p3-shuffle.pl [options]

This script reads a file in batches of 500,000 records at a time and writes them out in a different order. It is used to un-sort files for deep learning purposes.

Parameters

There are no positional parameters.

The standard input can be overridden using the options in Input Options.

Additional command-line options are as follows.

• batchSize

Number of records to read in a batch. The default is 500,000.

• verbose

If specified, progress messages will be written to STDERR.