Aceshardware

This is kind of long, but please read. I wouldn't believe this unless I got bitten by it - that Windows cannot successfully copy a large number of files.

In short, I've found that if you try to copy too many files in Windows XP or Server 2003 (32-bit for either), the machine becomes unstable and requires a reboot. Too many means as little as 760k files in WinXP, or about 2.8 million files in Server 2003. Additionally, this issue may be limited to machines with either multiple cores/cpus, or with Intel CPUs.

I have a business where I process large numbers of documents; I return the processed files on an external hard drive. We recently started updating computers here, and as part of our qualification process, we have a test suite of about 3MM (very small) files that we copy to make sure that things are working properly. We use robocopy for the test, but we've seen the same problem using other command line tools, as well as our own custom VB6 utility.

The symptoms of this problem are that the copying operation is interrupted with the following error:

ERROR 1450 (0x000005AA)

At that point, I find that network connections are unreliable or dropped, and other software either acts abnormally or freezes (errors include "Insufficient system resources exist to complete the requested service", or complaints of missing DLLs or other components that do exist).

The machine then requires a reboot to be useful again, and then behaves normally. The memory in each testbed has been passed at least 50 memtest passes, so the issue is not related to defective memory.

So far, our testing has shown the following:

+ AMD 940 socket single core CPU, XP32, Nvidia chipset: Successful copy operation.
+ AMD 939 socket single core CPU, Win2k, Nvidia chipset: Successfully copy operation.
+ Intel Q6600, Intel 975x chipset, XP32: FAILED at 760k files.
+ Intel Q6600, Intel i3210 chipset, XP32: FAILED at 760k files.
+ Intel Q6600, Nvidia 780i chipset, XP32: FAILED at 760k files.
+ Intel Q6600, Nvidia 780i chipset, XP64: Successful copy operation.
+ Intel Xeon 3070, Intel i3210 chipset, Server 2003R2: FAILED at 2.72mm files.

I am so far concluding that there is some internal memory issue in 32-bit Windows that only manifests with multiple CPUs/cores. I can reproduce this error easily; it must be something that commonly exists, but I can only find scant information about it via Google searches. We've tried the registry fixes mentioned here but with no effect - so it appears that our issue is different than what MS has seen in the past.

Has anyone seen anything like this? Is there anything that can be done about it? I have a feeling that this issue could explain a lot of popular sentiment towards Windows, as the effect seems cumulative in nature.

I'm including a redacted screen-grab (had to obscure the file names for confidentiality purposes).

NVidia had an issue along similar lines during the time of their NF2 3xx release.

There is a MSKB article on this issue here.

See my original posting. It may be the same error, and the issues may even be related, but the registry fix from the MSKB article are not effective.

In this case, the aggregate file size isn't all that huge - about 80 gigs - but it seems to be the total number of files.

Have you used robocopy w/o asserting the backup priv?

you can turn around by doing and "xcopy *.* /s" if you really need to copy so many files.

who?

I'm not sure what you mean by that - what flags in Robocopy would be appropriate?

I may not have been clear - the issue affects ALL copying programs and utilities. This includes:

+ Robocopy
+ xcopy
+ "ls" from cygwin
+ A VB6 utility that we wrote ourselves
+ Drag 'n' drop in Explorer

wow! that is very weird , does it still fail even if you ask xcopy to skip the errors?
who?

Increase the SharedSection session resources here: (20480 is default in AMD64, 3078 in IA32) Bump the 768 background value to 2048. There's a good chance that will fix it.

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\SubSystems\system32\csrss.exe ObjectDirectory=\Windows SharedSection=1024,20480,768 Windows=On SubSystemType=Windows ServerDll=basesrv,1 ServerDll=winsrv:UserServerDllInitialization,3 ServerDll=winsrv:ConServerDllInitialization,2 ProfileControl=Off MaxRequestThreads=16

I have been having the same issue for over a year now. This is becoming more of a problem as we work on larger projects so I am determined to resolve the issue this week. This problem occurs for us when attempting to copy lots of files (usually running robocopy or xcopy, but also seen in at least 2 other programs). We have seen this on numerous machines, but I haven't been tracking processor and chipset. The problem appeared to have gotten worse when we started using the /3GB switch in the boot.ini ... but that may be the time we started processing bigger jobs so it could be coincidence. I run Windows XP on the computers that copy the 100's of thousands of files, and the closest MS KB article I could find was http://support.microsoft.com/kb/304101 which is written for Windows 2000 / 2003. I tried the first fix from that KB (PoolUsageMaximum = 40) but did not work. I have just implemented the 2nd fix (PagedPoolSize = 0xffffffff) and testing again tonight. If that does not work, then I will try the SharedSection fix as suggested by AtWork. If anybody has any other ideas, please share.

The PoolUsageMaximum = 40 in combination with PagedPoolSize = 0xffffffff did not work. I removed those 2 changes, and changed the SharedSection from the original value of 1024,3072,512 to a new value of 1024,3072,768 (full path is HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\SubSystems, key = "Windows"). I tried the copy againand it still failed.

Any other ideas?

I tried changing SharedSection again this time bumping it up to 1024,3072,1024. It still failed, then I changed it again to 1024,4096,1024 and it failed again. I then added the PoolUsageMaximum = 40 back in (since that did keep my paged kernel memory much lower) and kept the SharedSection at 1024,4096,1024 but it is still failing with the same errors which are:
robocopy = ERROR 1450 (0x000005AA) ... Insufficient system resources exist to complete the requested service
Then I get "The application failed to initialize properly (0xc0000142)" when trying to do almost anything after that (like pull up Task Manager or another command prompt, or trying to run computer management, or trying to run shutdown.exe from the existing command prompt, etc.)

One thing to do is to defrag the hard drive where the system is at so that you can set aside a larger contiguous space for system resources. Manually set the virtual swap space, too.

Have you tried xxcopy?
www.xxcopy.com


I found it a few years ago and prefer it to several other utilities for doing simple backups/copying. While I only use a small subset of its options it can do a LOT, so much so that there's shortcut parameters that combine several others, its basically xcopy on some sort of alien steriods.

The basic one is
xxcopy c:\src d:\dest /clone

this will scan the source and destination and keep them in sync, ie only copy new or changed files and remove deleted files. There's options for not removing the deleted files to.

Its fast, does a ton and its cheap. I'm trying to find if there's anything documented about copying a million+ files but I'm half asleep right now.

DD