| Aceshardware http://aceshardware.freeforums.org/ |
|
| I think I've uncovered a major issue in Windows http://aceshardware.freeforums.org/i-think-i-ve-uncovered-a-major-issue-in-windows-t458-15.html |
Page 2 of 2 |
| Author: | retnuh [ Fri Apr 25, 2008 6:00 am ] |
| Post subject: | doing a little more digging |
pulled this part from the xxcopy forum, if nothing else here's a possible solution that'll batch the files in limited sets. Quote: I suggest the following:
xxcopy \mydir\ /aa Set the archive bit to all the files in the directory first. And then, run "incremental backup" using the /M switch. xxcopy \mydir\ \dest\ /m If the program somehow hangs, you can restart the same command which will not repeat the files that have already been successfully copied (so, you will make a progress every time you run it). But, I would like to suggest this in combination of an alternative technique to what Garry suggested. Use the Quota feature (/QF or /QBL ). xxcopy \mydir\ \dest\ /m /qf100k // 100,000 files Or, xxcopy \mydir\ \dest\ /m /qbl5GB // up to 5 GB These commands will limit the total number of files (or # of total bytes) to the specified value. You can make a loop in a batch file, (or just repeat the same lines for simplicity). Or, you can run an experiment by changing the /QF (or /QBL) parameter to see if the problem is really related to the total job size (start small and double it in each successive runs). At least, you will know where or not the total job size matters (I doubt it, though). |
|
| Author: | TXRanger [ Sun Apr 27, 2008 3:07 am ] |
| Post subject: | |
After spending waay too much time on this, I do have some additional information. The short version is that this appears to be fixed in Windows XP SP3 (at least v.3311). I created a batch file that would create a bunch of small files (it creates 3 million small files in about 13 minutes with a total size of about 212 MB). I then installed a bunch of computers in our lab with fresh copy of Windows XP that had SP2 built in. I then used the batch file to run a bunch of tests. Different computers would error at different times. Most of the computers would fail just running the batch file to create the files. Those that were able to create all 3 M files without running out of resources all failed running xcopy and all failed running robocopy to copy the files to an external hard drive. I found some upgraded storage drivers for one set of machines (for the Intel Storage Matrix on 945 chipset) that got the copy a lot further on the computers with that hardware, but still failed before 1 million files. After trying a bunch of fixes and updates I tried upgrading to Windows XP Service Pack 3 ("WindowsXP-KB936929-SP3-x86-ENU.exe" v.3311 is what we have in our lab) and it worked. So I upgraded the other 7 machines to XPSP3 and then they all passed also. Now all 8 are able to create all 3 million files with the batch file successfully and also able to robocopy (haven't tested xcopy or others yet but I suspect they will all work no) to an internal drive and external drive with write caching disabled and external drive with write caching enabled. I am processing a real-life project on one of the machines right now, so I have my fingers crossed that it will complete over the weekend without errors. I hope this information helps anybody else running into this problem. |
|
| Author: | TXRanger [ Mon Apr 28, 2008 5:49 am ] |
| Post subject: | |
Our processing job is still running over 28 hours into the job. Before the SP3 install, it had crashed several times and had not made it nearly this far. So according to our tests, this issue has been resolved by Microsoft in Windows XP SP3 (not sure about Windows 2003 or other operating systems). Please note that as of today Windows XP SP3 is still a beta, so install it at your own risk. Microsoft has "promised" the official SP3 release in the 1st half of 2008, and I hear the goal for the final version is April 29, just 2 days away. If anybody wants the batch file that creates the "millions of files" just let me know. It has variables to allow you to specify the number of files (directories and files per directory) and size of the files so you can change the variables to meet your testing needs. |
|
| Author: | pggsb [ Thu May 22, 2008 11:54 pm ] |
| Post subject: | |
TXRanger, Thanks so much for your information. This problem has been bothering me for quite some time and there was no solution before your discovery. Thanks again! |
|
| Author: | Forgotten [ Sat May 24, 2008 6:14 pm ] |
| Post subject: | |
I came here by google search..Because I get this error trying to install service pack 3 for winXP..But its a registry key its failing on. 
I have nothing else running, nothing open. I can find no reason I should be getting this error yet if others were getting this error on plain old vanilla windows xp on trying to install sp3 I would think I'd find more on the net than this and old win2k references. ARG. w t f |
|
| Author: | TacoBell [ Sun May 25, 2008 7:08 pm ] |
| Post subject: | |
This issue doesn't seem to affect x64 windows (x64 Vista SP1 to x64 2008) robocopy output, simple /MIR of 4.2 million files. Code: Total Copied Skipped Mismatch FAILED Extras Dirs : 4180 2498 1682 0 0 0 Files : 4262490 4020418 242072 0 0 172 Bytes : 258.903 g 142.375 g 116.528 g 0 0 12.76 m |
|
| Page 2 of 2 | All times are UTC + 1 hour |
| Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group http://www.phpbb.com/ |
|