Quick Fix: File count of a directory

Recently I had put together a DOS batch script that would monitor the number of PDFs in a directory. Once that count got to or above 500 it would then turn on some services to process the PDFs. Now there's probably numerous ways to determine the number of files in a given path, but I wanted to do it in a single line and ideally exercise existing DOS functionality where possible. That's how I came up with the following single line command chain.

for /f "tokens=1" %%A in ('dir %pdfDir%\*.pdf 2^>nul ^| find /i "File(s)" ^|^| echo 0') do (set fileCount=%%A)

First, let me explain why all the carrots “^” are present. If you didn't know, this is how you escape special characters, like pipes “|” and redirects “>”  in DOS. Since we're using the FOR command, we want to escape these characters because the FOR command will take the entire string given and run it internally to process the output. Before this happens though DOS will read the entire line, process/parse any variables and/or special characters, like pipes and redirects, and then run the command. This parsing can get confused when it sees the pipes and redirects without carrots and thinks you'll want to execute these actions before passing them to the FOR command. This is the same as to why you have to use double percent signs “%%” in a script for the FOR command, but not when using it on the command line. The double percent signs is how you escape percent signs so the DOS batch file doesn't treat it like a variable it should process, but let's the FOR command process it instead.

C:\TechBlog>echo Redirect = ^>  Pipe = ^|  Double pipe = ^|^|
Redirect = >  Pipe = |  Double pipe = ||

If you don't understand the “2>nul” syntax, please see my earlier post “Redirection of the output streams in DOS scripts”.

So let's break down what's happening here. I'm going to simplify the full command to the following to help explain what's going on and later describe why the “echo 0” is present.

for /f "tokens=1" %%A in ('dir %pdfDir%\*.pdf 2^>nul ^| find /i "File(s)"') do (set fileCount=%%A)

When you do a DIR on a directory you get output like the following.

C:\TechBlog>dir
 Volume in drive C has no label.
 Volume Serial Number is A48F-FE2B

 Directory of C:\TechBlog

04/21/2013  12:00 PM    <DIR>          .
04/21/2013  12:00 PM    <DIR>          ..
04/21/2013  12:00 PM               391 PdfFile1.pdf
04/21/2013  12:00 PM               444 PdfFile2.pdf
04/21/2013  12:00 PM               338 TextFile.txt
               3 File(s)          1,173 bytes
               2 Dir(s)  16,957,591,552 bytes free

Most of this information we don't care about except for the line stating the file count. We want to isolate this line so that we can use the FOR command to parse it. To do this we pipe the output of the DIR into the FIND command. The FIND is setup to only return lines that have the text “File(s)” in them. This line of text is returned and the FOR command parses it. Because we're using the FOR command with “tokens=1” we're saying that after the FOR command parses the line, which by default is by whitespaces, we only want the first token found. That token will be the number of files. So now when the FOR command runs the command (set fileCount=%%A) it's setting fileCount to the number of files found in the directory. We have our file count in a variable and ready to use!

Ah, but there's a catch. There's always a catch. Sadly when you do a DIR on something specific, like “dir *.doc” and there are no DOCs present it doesn't simply return “0 File(s)”, but the message “File Not Found” which isn't really useful for what we're trying to accomplish. We need some error handling.

C:\TechBlog>dir *.doc
 Volume in drive C has no label.
 Volume Serial Number is A48F-FE2B

 Directory of C:\TechBlog

File Not Found

This is where the “echo 0” comes in. Going back to the original full command, when DIR doesn't pipe any text into FIND that contains “File(s)”, the FIND command returns a failed (non-zero) exit code. Adding “||” to the end of a command means only run the following command if the previous has failed. Since FIND has failed, the “echo 0” command is ran returning the text “0” which FOR then parse. This then resolves our “File Not Found” scenario and we're good to go!

Do you know of another single line command/command chain to determine the number of files in a path? If so, share it! I'd love to see how others have accomplished this.

Quick Fix: “Invalid number” error in DOS batch script

I ran into this the other day. After a little searching around I found the answer and thought I'd share the way I resolved it. Hopefully this will shorten someone else's search time.

The error message I was getting was the following: 

Invalid number. Numeric constants are either decimal (17),
hexadecimal (0x11), or octal (021).

It turns out this is happening on the script line that's doing some math with the “set /a” command. It could be happening in other commands as well, but this is where I've seen it.

So here's what you need to know to resolve this. As the error message indicates, it treats values that have leading 0's as octal. Each digit of a octal value is represented with the numbers 0-7. For example octal value 022 is the same as decimal value 18. Why you're getting the error message is because you have a value that has a leading zero, indicating an octal value, but contains at least a digit that is either an 8 or 9. Neither number conforms with how an octal value is represented and this is the source of the error.

Here's the quick solution I came up with. Since I was dealing with dates (months and nth day of the month) I knew that I would have values that are at most 2 digits long and always positive. So before doing any arithmetic on these values I would do a quick check for a leading zero and remove it if present. 

@echo off
(set month=08)
echo The month should be 08: %month%
 
REM The error message should appear. Bummer!
(set /a nextMonth=%month% + 1)
echo The next month should be 9: %nextMonth%
 
REM Checking for leading zero and removing it if present.
if "%month:~0,1%"=="0" (set month=%month:~1%)
echo The month should now be 8: %month%
 
REM No error message. Hooray!
(set /a nextMonth=%month% + 1)
echo The next month should be 9: %nextMonth% 

To break it down, what's happening in the first part of the IF statement is we're obtaining a substring from month whose index starts at 0 and we want the substring to be 1 character long. This means that the code “%month:~0,1%” is just returning the first character of month, which in this case is a zero. Because the IF statement is true, we now set the value of month so that it equals the substring starting from the index of 1 and then all following characters. This means that “%month:~1%” will return the 8. And now the leading zero has been removed and we're good to go!

If you know you're values are only going to be 2 digits long, you could get away with an IF statement like the following:

if %month% lss 10 (set month=%month:~-1%)

I prefer the substring method I initially showed because you could later add some looping logic for removing multiple leading zeros if you were dealing with values like 00789.

Remember, this solution does not handle negative values or values that have more than 1 leading zero. Please take that into consideration when implementing this workaround in your script.

I hope this helps unblock you and clears up any issues you may have been having. If you have another way you handle related scenarios, post it in the comments. I'd love you hear other ways to accomplish this!

Redirection of the output streams in DOS scripts

Just like most programming languages, DOS batch files provides you access to 2 types of output streams. The standard out (STDOUT) and standard error (STDERR). STDOUT is where most of your output is directed to. Whereas all error messages should be directed to the STDERR. 

So, before getting into explaining how to use these different streams, you should probably know how DOS presents them. When doing redirection batch files use the number 1 to represent STDOUT and the number 2 to represent STDERR. By default, redirection is capturing the STDOUT stream. So the following statements both capture the STDOUT. 

C:\>dir *.* > output.txt
C:\>dir *.* 1> output.txt

Now, if you're not familiar with the STDERR stream, the following displays how programs output to this stream and not the STDOUT.

C:\>dir NonExistentFile.txt > output.txt
File not found.

See how even though you captured the output, the text “File Not Found” was displayed? This is because it's coming from the STDERR stream and not the STDOUT. If you were to open the output.txt file you see that it's empty since nothing was sent to the STDOUT stream. Now if you wanted to capture the error stream, you could do something like the following examples.

C:\>dir /b NonExistentFile.txt 2> error.txt
C:\>dir /b NonExistentFile.txt 2> error.txt > output.txt

For both examples the error.txt file will contain the text “File Not Found”. The second examples is just illustrating that you can capture both streams to separate files in the same command. This can be useful if you want to capture all the errors to one file and all the other text to another file. But if you want to capture all output to just 1 file, you could do something like this.

C:\>dir /b * > allOutput 2>&1
C:\>dir /b NonExistentFile.txt > allOutput.txt 2>&1

In the first example, all the files and directories found in the C:\ will be listed in allOutput.txt. Whereas the second capture the error message “File Not Found” into allOutput.txt. What's happening here is that 2>&1 is telling the script to direct STDERR stream into STDOUT. This way you're redirecting all output into 1 file. This is very useful for recording information for debugging. You could direct the STDOUT to STDERR similarly by using 1>&2, but this is uncommon.

So far we've only shown examples of where we redirect STDOUT and STDERR to a file. What if we don't want to record the output to a file and would just rather redirect it into an oblivion? Well DOS reserves some file names for special uses. Among them is NUL which represents a null file. Linux scripters will recognize this as /dev/null. When you write to this file the content just disappears into the ether and is never to be seen again. I find this useful testing conditions and don't care about the output. Below are a couple of examples.

if exist logFile.txt del /f /s logFile.txt > NUL

where powershell.exe > NUL 2>&1
if ERRORLEVEL 1 (
   echo ERROR: PowerShell is not installed; stopping.
   goto :eof
)

In the first line I'm redirecting the message “Deleted file – logFile.txt” to NUL since this information has no importance to me. In the following section I don't care about any of the output text from the WHERE command is, I'm just interested in the error code it returns. So I redirect all output to NUL.

Hope you now have a better understanding of how the STDOUT and STDERR streams work in DOS and how they can give you a new level of control over your scripts. If you have any tips or tricks related to these, please post them in the comments below! It would be great to learn what other people are doing with them.

Handy Hack: Minimize A Running Batch File Window

Every some often we all come across a problem that there's no real good way to solve. In which case we have to come up with a workaround. They're not always pretty. Sometimes we're surprised they even work at all. We don't even try to gussy them up because putting lipstick on a Frankenstein doesn't make them any less scary. We just call them what they are.

They're hacks.

One reason I started this blog was to post useful things that I came across whether it be hacks, an algorithm, or just some advice. I want this to be the place that has the information I couldn't easily find or has a more in depth explanation on how something works. And with saying that, I'm going to start posting more short entries starting with this one.

Like most random useful tricks and hacks, I figured this out while investigating a completely different problem. I've never found an automated way to minimize, or maximize, a running DOS batch file window. I know I've seen other people searching for methods of doing this as well because I've seen their posts on various forums while searching myself. So to my surprise I ran across this while debugging a PowerShell issue my team was having.

Yes. This does require PowerShell to be installed on your machine. This is a hack. They're not always elegant and they often rely on the behavior of something outside of what you would've expected or wanted to rely on. But to the best of my knowledge PowerShell is installed as part of Vista and up. So it's not reaching too out of the norm to pull this off.

If you look at the options for PowerShell you'll see that there's a -WindowStyle flag. The choices for it are Normal, Minimized, Maximized, and Hidden. Using this flag when you start PowerShell will place the window in the selected state.

“Ok,” you may be saying to yourself, “That's great, but now my batch file is blocked until I quit PowerShell.” And this is true. But PowerShell also has the -Command flag which allows you to run a command and then exit when finished. Why not use this flag to execute the “exit” command so PowerShell quits immediately?

Folks, we have ourselves a hack. Now shake your head in disbelief that someone suggested it, give a sigh of defeat for lack of a better solution, and then copy and paste it into your script.

Below are examples of how to do this which can be ran from a DOS prompt. I'm also using -NoLogo just because it makes things cleaner and -NoProfile to speed up the loading of PowerShell. The ordering doesn't matter exception for -Command which must be the last flag. There's also the -NonInteractive flag. I haven't played with this too much and since PowerShell is only running briefly, it's probably not needed, but doesn't hurt either.

To minimize the window:

C:\>PowerShell.exe -NoLogo -NoProfile -WindowStyle Minimize -Command exit

To maximize the window:

C:\>PowerShell.exe -NoLogo -NoProfile -WindowStyle Maximize -Command exit

To return the window to normal (its default size):

C:\>PowerShell.exe -NoLogo -NoProfile -WindowStyle Normal -Command exit

This next one you have be careful with. It will hide your window, remove it from the Applications tab in Task Manager (it does still shows up under the Processes tab), and at least on Windows 7, no longer show up on the task bar. I haven't played with this enough to figure how to make it visible again. You have been warned.

To hide the window:

C:\>PowerShell.exe -NoLogo -NoProfile -WindowStyle Hidden -Command exit

Now these are shown using the command line, but you can just drop them into a batch file to get the same effect. I added the minimize command to an existing script that's launched by Task Scheduler as me. This way I no longer have a black box jumping out of nowhere, breaking my concentration, causing alarm while I determine what's happening, and then realize it's just copying files so I minimize it.

It's the little things that make life grand.

Happy scripting!

Variable Expansion Within a DOS FOR Loop

Ah, the DOS FOR loop. Had I known about you earlier in life, I would probably have a day or two back. But atlas, it wasn't until recently that we met.

My first tech blog post and I'm talking about a FOR loop in DOS? Well, I figured I should start at the beginning, and for me, that's DOS.

In my current contract, I do a lot of automation in DOS. And any kind of automation is made easier with loops. Enter the FOR command. I have to admit that I didn’t know anything about the FOR command under DOS before I started this job and had to analyze someone else’s batch files. But now that I have, I’m a better scripter for knowing it.

One of the more troublesome things that I’ve dealt with is trying to set environment variables and using them all within the FOR loop. It just doesn’t work. But if you stop to think about it, it makes sense as to why it doesn’t work. Look at the example below.

Script

SET check=0
FOR %%A IN (1 2) DO (
   SET check=%%A
   ECHO Mic check %check%.
)

Output

Mic check 0.
Mic check 0.

Because all of the environment variables within the DO () are expanded only when the FOR command is reached, not for every iteration of the loop, their values are static. So in this example when the FOR command is reached, the expansion of check to its current value of 0 takes place and then the looping begins. It would look something this.

SET check=0
FOR %%A IN (1 2) DO (
   SET check=%A
   ECHO Mic check 0.
)

Note how check is expanded to 0 inside the DO ().

So, how do you deal with this limitation? There are 2 ways. The first is by using the CALL command to move the logic out of the DO () and to a different part of your script. Below is a modified version of the script that behaves in the wanted manner.

Script

SET check=0
FOR %%A IN (1 2) DO (
   CALL :MicCheck %%A
)
GOTO :eof

:MicCheck
SET check=%1
ECHO Mic check %check%.
GOTO :eof

Output

Mic check 1.
Mic check 2.

The second way of handling this is by turning on delayed variable expansion. I don’t like this method as much because unless you have it enabled by default, which it’s not as of the time of this posting, scripts don’t work out-the-door. For them to work on first run, you have to play with the SETLOCAL and ENDLOCAL commands inside your batch file. I’ve run into odd scope behaviors in the past doing this, so I prefer the CALL method shown above.

Script

SET check=0
SETLOCAL ENABLEDELAYEDEXPANSION

FOR %%A IN (1 2) DO (
   SET check=%%A
   ECHO Mic check %check%.
)

ENDLOCAL

Output

Mic check 1.
Mic check 2.

Now, this is a simple example, but I’m sure you can see how you can expand this to do what you’d like.

Happy scripting!