Dir /b /s different in cmd.exe and batch file

I want to know why there is a delay between dir /b/s C:\*.* in cmd.exe and batch file.

I tried the blow batch file, But it takes about one hour to show me the result, but dis /b/s in cmd.exe show the result fast.

for /f "tokens=*" %%a in ('dir/b/s c:\*.*') do (
echo "%%a" 

copy "%%a" C:\windows\ )

Please help me to show the result in batch file fast like cmd.exe.

Answer

There are two elements that lead to this behaviour

• for /f will always retrieve all the data that it needs to process before starting to process it. Than means that for /f will "sit" (not execute the code in the do clause) while dir works, waiting for all the data.


• When for /f reads a disk file it will "simply" acomodate a buffer large enough to load it into memory, load the file and start processing it. But when the data source is a command execution, not knowing the final size of the data, a buffer is defined and resized as needed while retrieveing the command's output.

The need to retrieve all the data and the process of resizing the buffer is what generates the delay.

Why? As a example:

• If I use dir /s /b c:\windows I get a list of 119343 files, 13MB of data.


• As the memory buffer defined by for /f starts at 4KB and is increased in 4KB each time it is full it will need 3327 resize operations.


• Each time a resize is needed, a new 4KB larger buffer is allocated and the data inside the old buffer is copied into the new larger buffer. For 13MB we need 3327 resize operations which means aprox. 21GB in memory copy operations (data to copy increases each time the buffer is resized). Maybe it does not seem a lot and memory is fast, but sometimes (ex. here) things are not so simple.

If you add the time needed to retrieve the data from disk to the time needed to handle the memory allocation/memory copy, before starting to process the data, you have a visible delay.

If you need remove the delay, don't use a for /f. A better option (while keeping a similar approach) could be

for /r "c:\" %%a in (*) do (
    echo "%%~fa"
)

That is, use the recursive version of the for command, from the indicated starting folder.