Randomly select a line out of a text file and set that line as a variable

Question

How to randomly select a line/ip out of a text file, and set it as a variable.

I mean something like:

set IP=TheLine/IPFromOfTheFile

Answer 1

The array based solutions seem like they ought to be fastest because they only read the file once. But variable manipulation becomes very slow when the environment becomes very large. So the array based solutions give non-linear performance. They can be painfully slow if the file is very large.

For a pure batch based solution that gives decent linear performance, regardless of file size, I like aschipfl's answer. However, it has problems with lines that begin with ; due to the default for /f EOL value. It also fails if the file is empty. Here is a corrected version that is nearly perfect.

@echo off
setlocal
set "file=test.txt"
set "LINE="
for /F %%N in (
  'type "%file%" ^| find /C /V ""'
) do set set /A RND=%RANDOM% %% %%N 2>nul || goto :BREAK
if %RND%>0 (set "skip=skip^=%RND%") else set "skip="
for /F usebackq^ %skip%^ delims^=^ eol^= %%L in ("%file%") do (
  set "LINE=%%L"
  goto :BREAK
)
:BREAK
set LINE

There is one remaining flaw with the above - If the selected line is empty, then it will return the next non-empty line instead due to the design of FOR /F.

The "standard" way to fix that is to use FINDSTR to prefix each line with the line number, and then use expansion find/replace to remove the prefix. Given the presence of the line number prefix, I like to use an additional FINDSTR to select the correct line, rather than relying on FOR /F

@echo off
setlocal
set "file=test.txt"
set "ln="
for /f %%N in (
  'type "%file%" ^| find /c /v ""'
) do set /a rnd=%random% %% %%N + 1 2>nul || goto :result
for /f "delims=" %%L in (
  'findstr /n "^" "%file%" ^| findstr "^%rnd%:"'
) do set "ln=%%L"
setlocal enableDelayedExpansion
set "ln=!ln:*:=!"
:result
set ln

If you know that all your lines are <= 1021 bytes long, then there is a simpler solution for dealing with empty lines that reads the file with SET /P instead of FOR /F.

@echo off
setlocal
set "file=\utils\jrepl.bat"
set "ln="
for /f %%N in (
  'type "%file%" ^| find /c /v ""'
) do set /a rnd=%random% %% %%N 2>nul || goto :result
<"%file%" (
  for /l %%N in (1 1 %rnd%) do set /p "ln="
  set "ln="
  set /p "ln="
)
:result
set ln

Just for yucks, I decided to write a solution based on my JREPL.BAT regular expression text processing utility. An optimized hybrid JScript/batch script could be written, but JREPL is convenient if you already have it in your bag of tricks.

If all you need is to print a random line, then it is a simple one liner:

call jrepl "^.*" $0 /c /jmatch /jbeg "rnd=Math.floor(Math.random()*cnt)+1" /jbegln "skip=(rnd!=ln)" /f test.txt

A bit more code is needed to get the value in a variable:

@echo off
setlocal
set "ln="
for /f "delims=" %%L in (
  'jrepl "^.*" $0 /c /n 1 /jmatch /jbeg "rnd=Math.floor(Math.random()*cnt)+1" /jbegln "skip=(rnd!=ln)" /f test.txt'
) do set "ln=%%L"
setlocal enableDelayedExpansion
set "ln=!ln:*:=!"
:result
set ln

If the lines are all <= 1021 and you don't mind a temp file, then you could use SET /P

@echo off
setlocal
call jrepl "^.*" $0 /c /jmatch /jbeg "rnd=Math.floor(Math.random()*cnt)+1" /jbegln "skip=(rnd!=ln)" /f test.txt /o temp.txt
set "ln="