using START to try and parallelize copy operations, getting hung u

[Mark]

Hi...

I have a batch file I'm launching from a web page using psexec.exe. The
last stage of the batch file is to copy a file to several locations. The
problem is that 2 of the 3 destinations are going through a network pipe the
size of a swizzle stick.

I was hoping to cut down on the pain by parallelizing the copies with START
(i.e. move the actual XCOPY command into its own little bat and have the
parent bat start them all at once).

I figured that the bottleneck is the network pipe, not the disk access to
the local file being copied so I might save some time by getting all the
copies going at once.

Problem is that when I ran this up the flagpole, the parent bat just hung at
the start command. Didn't copy anything. Is START something you shouldn't
use when you're already in the background? Or running under psexec?

Thanks
Mark

[Pegasus [MVP]]

Post by Mark
Hi...
I have a batch file I'm launching from a web page using psexec.exe. The
last stage of the batch file is to copy a file to several locations. The
problem is that 2 of the 3 destinations are going through a network pipe the
size of a swizzle stick.
I was hoping to cut down on the pain by parallelizing the copies with START
(i.e. move the actual XCOPY command into its own little bat and have the
parent bat start them all at once).
I figured that the bottleneck is the network pipe, not the disk access to
the local file being copied so I might save some time by getting all the
copies going at once.
Problem is that when I ran this up the flagpole, the parent bat just hung at
the start command. Didn't copy anything. Is START something you shouldn't
use when you're already in the background? Or running under psexec?
Thanks
Mark

This is probably easy to work out but my crystal ball is a little cloudy
right now. I've booked it for a cut & polish service early next week. Care
to post your batch file in the meantime?

[Mark]

The whole system is too large to post here, but the part I'm asking about is
a bat file spawned from a javascript .wsf file:
ECHO Copy call for %2
time /t
@IF NOT DEFINED StageServer GOTO END
start /b /high Copy.Machine1.bat %1 %2
start /b /high Copy.Machine2.bat %1 %2
start /b /high /wait cmd /c Copy.Machine3.bat %1 %2
time /t
:END

And each of the Copy bat files looks like this:
@ECHO Starting Machine001 copy
time/t
@NET USE \\Machine1\ /user:Machine1\%username%
XCOPY /Y "%1" "\\Machine1\%2"
@ECHO Done with Machine1 copy
time/t

Using Procexp.exe, I can see that the bat file is hung on the first start
command and never finishes. Nor is any file copied to Machine1.

Thanks
mark

Post by Pegasus [MVP]

Post by Mark
Hi...
I have a batch file I'm launching from a web page using psexec.exe. The
last stage of the batch file is to copy a file to several locations. The
problem is that 2 of the 3 destinations are going through a network pipe the
size of a swizzle stick.
I was hoping to cut down on the pain by parallelizing the copies with START
(i.e. move the actual XCOPY command into its own little bat and have the
parent bat start them all at once).
I figured that the bottleneck is the network pipe, not the disk access to
the local file being copied so I might save some time by getting all the
copies going at once.
Problem is that when I ran this up the flagpole, the parent bat just hung at
the start command. Didn't copy anything. Is START something you shouldn't
use when you're already in the background? Or running under psexec?
Thanks
Mark

This is probably easy to work out but my crystal ball is a little cloudy
right now. I've booked it for a cut & polish service early next week. Care
to post your batch file in the meantime?

[Pegasus [MVP]]

To find out where your batch file gets stuck, use the syntax below, then
examine the log file c:\test.txt. It probably tells you everything you need
to know.

@echo off
ECHO Copy call for %2 on %date% at %time% (User=%UserName%) >> c:\Test.txt
IF NOT DEFINED StageServer GOTO :eof
start /b /high c:\Tools\Copy.Machine1.bat %1 %2
start /b /high c:\Tools\Copy.Machine2.bat %1 %2
start /b /high /wait cmd /c c:\tools\Copy.Machine3.bat %1 %2
ECHO Copy call for %2 on %date% at %time% >> c:\Test.txt

@echo off
ECHO Starting Machine001 copy on %date% at %time% >> c:\Test.txt
NET USE \\Machine1\ /user:Machine1\%username% 1>> c:\Test.txt 2>>&1
XCOPY /Y /c "%1" "\\Machine1\%2" 1>> c:\Test.txt 2>>&1
ECHO Done with Machine1 copy on %date% at %time% >> c:\Test.txt

Post by Mark
The whole system is too large to post here, but the part I'm asking about is
ECHO Copy call for %2
time /t
@IF NOT DEFINED StageServer GOTO END
start /b /high Copy.Machine1.bat %1 %2
start /b /high Copy.Machine2.bat %1 %2
start /b /high /wait cmd /c Copy.Machine3.bat %1 %2
time /t
:END
@ECHO Starting Machine001 copy
time/t
@NET USE \\Machine1\ /user:Machine1\%username%
XCOPY /Y "%1" "\\Machine1\%2"
@ECHO Done with Machine1 copy
time/t
Using Procexp.exe, I can see that the bat file is hung on the first start
command and never finishes. Nor is any file copied to Machine1.
Thanks
mark

Post by Pegasus [MVP]

Post by Mark
Hi...
I have a batch file I'm launching from a web page using psexec.exe.
The
last stage of the batch file is to copy a file to several locations.
The
problem is that 2 of the 3 destinations are going through a network
pipe
the
size of a swizzle stick.
I was hoping to cut down on the pain by parallelizing the copies with START
(i.e. move the actual XCOPY command into its own little bat and have the
parent bat start them all at once).
I figured that the bottleneck is the network pipe, not the disk access to
the local file being copied so I might save some time by getting all the
copies going at once.
Problem is that when I ran this up the flagpole, the parent bat just
hung
at
the start command. Didn't copy anything. Is START something you shouldn't
use when you're already in the background? Or running under psexec?
Thanks
Mark

This is probably easy to work out but my crystal ball is a little cloudy
right now. I've booked it for a cut & polish service early next week. Care
to post your batch file in the meantime?

[Mark]

The output was already being captured farther up. It gets as far as the
start command, where it hangs. Nothing in the bat to be executed comes out,
nothing after the first start command.

Post by Pegasus [MVP]
To find out where your batch file gets stuck, use the syntax below, then
examine the log file c:\test.txt. It probably tells you everything you need
to know.
@echo off
ECHO Copy call for %2 on %date% at %time% (User=%UserName%) >> c:\Test.txt
IF NOT DEFINED StageServer GOTO :eof
start /b /high c:\Tools\Copy.Machine1.bat %1 %2
start /b /high c:\Tools\Copy.Machine2.bat %1 %2
start /b /high /wait cmd /c c:\tools\Copy.Machine3.bat %1 %2
ECHO Copy call for %2 on %date% at %time% >> c:\Test.txt
@echo off
ECHO Starting Machine001 copy on %date% at %time% >> c:\Test.txt
NET USE \\Machine1\ /user:Machine1\%username% 1>> c:\Test.txt 2>>&1
XCOPY /Y /c "%1" "\\Machine1\%2" 1>> c:\Test.txt 2>>&1
ECHO Done with Machine1 copy on %date% at %time% >> c:\Test.txt

Post by Mark
The whole system is too large to post here, but the part I'm asking about is
ECHO Copy call for %2
time /t
@IF NOT DEFINED StageServer GOTO END
start /b /high Copy.Machine1.bat %1 %2
start /b /high Copy.Machine2.bat %1 %2
start /b /high /wait cmd /c Copy.Machine3.bat %1 %2
time /t
:END
@ECHO Starting Machine001 copy
time/t
@NET USE \\Machine1\ /user:Machine1\%username%
XCOPY /Y "%1" "\\Machine1\%2"
@ECHO Done with Machine1 copy
time/t
Using Procexp.exe, I can see that the bat file is hung on the first start
command and never finishes. Nor is any file copied to Machine1.
Thanks
mark

Post by Pegasus [MVP]

Post by Mark
Hi...
I have a batch file I'm launching from a web page using psexec.exe.
The
last stage of the batch file is to copy a file to several locations.
The
problem is that 2 of the 3 destinations are going through a network
pipe
the
size of a swizzle stick.
I was hoping to cut down on the pain by parallelizing the copies with START
(i.e. move the actual XCOPY command into its own little bat and have the
parent bat start them all at once).
I figured that the bottleneck is the network pipe, not the disk access to
the local file being copied so I might save some time by getting all the
copies going at once.
Problem is that when I ran this up the flagpole, the parent bat just
hung
at
the start command. Didn't copy anything. Is START something you shouldn't
use when you're already in the background? Or running under psexec?
Thanks
Mark

This is probably easy to work out but my crystal ball is a little cloudy
right now. I've booked it for a cut & polish service early next week. Care
to post your batch file in the meantime?

[Pegasus [MVP]]

Here are a couple of pointers:
- Terminate each of your Copy.MachineX.bat files with an explicit "exit"
command.
- The syntax for the Start command is:
START ["Title"] [/D Path] [/I] [/MIN] [/MAX] [/SEPARATE | /SHARED] . . .
The "Title" parameter is compulsory when the program name you invoke has
embedded spaces and is surrounded by double quotes.

Post by Mark
The output was already being captured farther up. It gets as far as the
start command, where it hangs. Nothing in the bat to be executed comes out,
nothing after the first start command.

Post by Pegasus [MVP]
To find out where your batch file gets stuck, use the syntax below, then
examine the log file c:\test.txt. It probably tells you everything you need
to know.
@echo off
ECHO Copy call for %2 on %date% at %time% (User=%UserName%) >> c:\Test.txt
IF NOT DEFINED StageServer GOTO :eof
start /b /high c:\Tools\Copy.Machine1.bat %1 %2
start /b /high c:\Tools\Copy.Machine2.bat %1 %2
start /b /high /wait cmd /c c:\tools\Copy.Machine3.bat %1 %2
ECHO Copy call for %2 on %date% at %time% >> c:\Test.txt
@echo off
ECHO Starting Machine001 copy on %date% at %time% >> c:\Test.txt
NET USE \\Machine1\ /user:Machine1\%username% 1>> c:\Test.txt 2>>&1
XCOPY /Y /c "%1" "\\Machine1\%2" 1>> c:\Test.txt 2>>&1
ECHO Done with Machine1 copy on %date% at %time% >> c:\Test.txt

Post by Mark
The whole system is too large to post here, but the part I'm asking
about
is
ECHO Copy call for %2
time /t
@IF NOT DEFINED StageServer GOTO END
start /b /high Copy.Machine1.bat %1 %2
start /b /high Copy.Machine2.bat %1 %2
start /b /high /wait cmd /c Copy.Machine3.bat %1 %2
time /t
:END
@ECHO Starting Machine001 copy
time/t
@NET USE \\Machine1\ /user:Machine1\%username%
XCOPY /Y "%1" "\\Machine1\%2"
@ECHO Done with Machine1 copy
time/t
Using Procexp.exe, I can see that the bat file is hung on the first start
command and never finishes. Nor is any file copied to Machine1.
Thanks
mark

Post by Pegasus [MVP]

Post by Mark
Hi...
I have a batch file I'm launching from a web page using psexec.exe.
The
last stage of the batch file is to copy a file to several locations.
The
problem is that 2 of the 3 destinations are going through a network
pipe
the
size of a swizzle stick.
I was hoping to cut down on the pain by parallelizing the copies
with
START
(i.e. move the actual XCOPY command into its own little bat and have the
parent bat start them all at once).
I figured that the bottleneck is the network pipe, not the disk
access
to
the local file being copied so I might save some time by getting all the
copies going at once.
Problem is that when I ran this up the flagpole, the parent bat just
hung
at
the start command. Didn't copy anything. Is START something you shouldn't
use when you're already in the background? Or running under psexec?
Thanks
Mark

This is probably easy to work out but my crystal ball is a little cloudy
right now. I've booked it for a cut & polish service early next week. Care
to post your batch file in the meantime?

[Mark]

Not sure which one actually did the trick, but I added

start "title" /b cmd /s /c C:\tools\Copy,machine1.bat %1 %2

(adding the title and cmd /s /c in lieu of exit in the bat file)

And it worked!

Thanks for your help...
Mark

Post by Pegasus [MVP]
- Terminate each of your Copy.MachineX.bat files with an explicit "exit"
command.
START ["Title"] [/D Path] [/I] [/MIN] [/MAX] [/SEPARATE | /SHARED] . . .
The "Title" parameter is compulsory when the program name you invoke has
embedded spaces and is surrounded by double quotes.

Post by Mark
The output was already being captured farther up. It gets as far as the
start command, where it hangs. Nothing in the bat to be executed comes out,
nothing after the first start command.

Post by Pegasus [MVP]
To find out where your batch file gets stuck, use the syntax below, then
examine the log file c:\test.txt. It probably tells you everything you need
to know.
@echo off
ECHO Copy call for %2 on %date% at %time% (User=%UserName%) >> c:\Test.txt
IF NOT DEFINED StageServer GOTO :eof
start /b /high c:\Tools\Copy.Machine1.bat %1 %2
start /b /high c:\Tools\Copy.Machine2.bat %1 %2
start /b /high /wait cmd /c c:\tools\Copy.Machine3.bat %1 %2
ECHO Copy call for %2 on %date% at %time% >> c:\Test.txt
@echo off
ECHO Starting Machine001 copy on %date% at %time% >> c:\Test.txt
NET USE \\Machine1\ /user:Machine1\%username% 1>> c:\Test.txt 2>>&1
XCOPY /Y /c "%1" "\\Machine1\%2" 1>> c:\Test.txt 2>>&1
ECHO Done with Machine1 copy on %date% at %time% >> c:\Test.txt

Post by Mark
The whole system is too large to post here, but the part I'm asking
about
is
ECHO Copy call for %2
time /t
@IF NOT DEFINED StageServer GOTO END
start /b /high Copy.Machine1.bat %1 %2
start /b /high Copy.Machine2.bat %1 %2
start /b /high /wait cmd /c Copy.Machine3.bat %1 %2
time /t
:END
@ECHO Starting Machine001 copy
time/t
@NET USE \\Machine1\ /user:Machine1\%username%
XCOPY /Y "%1" "\\Machine1\%2"
@ECHO Done with Machine1 copy
time/t
Using Procexp.exe, I can see that the bat file is hung on the first start
command and never finishes. Nor is any file copied to Machine1.
Thanks
mark

Post by Pegasus [MVP]

Post by Mark
Hi...
I have a batch file I'm launching from a web page using psexec.exe.
The
last stage of the batch file is to copy a file to several locations.
The
problem is that 2 of the 3 destinations are going through a network
pipe
the
size of a swizzle stick.
I was hoping to cut down on the pain by parallelizing the copies
with
START
(i.e. move the actual XCOPY command into its own little bat and have the
parent bat start them all at once).
I figured that the bottleneck is the network pipe, not the disk
access
to
the local file being copied so I might save some time by getting all the
copies going at once.
Problem is that when I ran this up the flagpole, the parent bat just
hung
at
the start command. Didn't copy anything. Is START something you shouldn't
use when you're already in the background? Or running under psexec?
Thanks
Mark

This is probably easy to work out but my crystal ball is a little cloudy
right now. I've booked it for a cut & polish service early next week. Care
to post your batch file in the meantime?

[Pegasus [MVP]]

Post by Mark
Not sure which one actually did the trick, but I added
start "title" /b cmd /s /c C:\tools\Copy,machine1.bat %1 %2
(adding the title and cmd /s /c in lieu of exit in the bat file)
And it worked!
Thanks for your help...
Mark

Thanks for the feedback.