Softpanorama

May the source be with you, but remember the KISS principle ;-)
Home Switchboard Unix Administration Red Hat TCP/IP Networks Neoliberalism Toxic Managers
(slightly skeptical) Educational society promoting "Back to basics" movement against IT overcomplexity and  bastardization of classic Unix

Windows bulk file copy tools

News Filesystems Recovery Recommended Links   xcopy Robocopy Microsoft Power Toys
Cygwin tutorial Windows Resource Kits Windows Disk Protection Windows Powershell oneliners The History of Development of Norton Commander History of Norton Utilities
Undeleting files Windows  Recovery Console Norton Ghost Windows Tips Humor Etc

Windows has xcopy is a useful built-in bulk file copy tool. But in Windows XP it has limitations and can't copy, say 20K files from one disk to another. It just chokes.  So you need another took in case you often have to copy an entire disk. This defect was fixed in Windows 7. I successfully copied 32GB drives with 30K files using it.

Microsoft Robocopy

Microsoft's Robocopy, a command line directory replication command, gradually replaced the older xcopy. It rocks a whole host of features that make is the logical choice for transferring a large number of files. It's multi-threaded, has a ton of options, and generally outperforms vanilla Windows copy operations. Best of all, it's built right in to Redmond's operating system. Especially useful for network copy operations and backups, Robocopy doesn't stop to ask you one hundred questions while it copies over your music collection, either.

The reality of benchmarking file copy performance is that you need something fast to copy from and something fast to copy to. This is most important with SSDs. It doesn't matter if your drive can write sequentially at 500 MB/s if the source files are hosted on a USB 2.0-attached external hard drive. We're copying our test files from an Intel SSD DC S3700 to the drives in the chart below, taking source speed out of the equation.

There are 9065 files comprising the 16.2 GB payload. Some of the files are huge (up to 2 GB), while others are best described as tiny. On average, that's around 1.8 MB per file. The files are a mix of music, program, pictures, and random file types.

It's fair to say that this chart would look much different if we were copying from a hard drive to a SSD. Even if the disk drive's sequential throughput wasn't a bottleneck, it'd still choke on the smaller files.

 

There are several choices that can improve the situation. Among them:

  1. Xcopy. Windows 7 xcopy utility has improved capabilities to copy large number of files. 20K is not a problem anymore. With options /r and /d can also be used for synchronization:

    xcopy   y:\Public_html f:\Public_html /s /r /c /d
  2. Robocopy. robocopy is a standard windows utility which is present in Windows 7, Windows 8 and Windows 10.  Part of Windows server 2003 resource kit. Looks like a replacement of xcopy with better capabilities. Has Mirror mode /mir. In this mode  can sincronize large partitions.  See also Robocopy SS64.com

  3. XXCOPY. This is commercial tool which is also available for free for personal use. The latest version which I tried is 3.21.8.. It is a command like that preserves XCOPY syntax and  extend XCOPY functionality.

    When we designed XXCOPY, one of the key design principles was to  keep the command syntax and the behavior of XXCOPY as close to  those of the original XCOPY program.  Although we did not agree  with all the idiosyncrasies of XCOPY, modifying anything XCOPY  already implements would cause substantial confusion to the users.

    If we change things arbitrarily, any incremental improvements we may add would be more than offset by the burden of remembering the specifics of changes we make.  That would not benefit anyone.

    It performs file-copy, delete, search and many common file management operations. It can select files and directories by the date, age, and size, as well as the comparison between the source and destination directories. It accepts wildcards in any place and with any number in the source specifier. Its ability to preserve the SFN makes it ideal for volume cloning. 
    xxcopy online manual

    In my test it took approximately 40 min to clone 32GB rive drive with   approximately 20GB filled with 40K files.

    Example:

    XXCOPY.EXE f:\Public_html n:\Public_html /clone
  4. TeraCopy TeraCopy is the GUI tool giving you great speed, along with useful and interesting features. It is free for home users. But if you are a Pro user, you will have to buy the Pro version for $19.95.
  5. Unstoppable Copier: As the name suggests, it is the utility, when you expect a lot of files to be damaged or corrupt. It is best suited for salvaging whatever it can from a pool of damaged files.

Semi-suppoted tools

  1. Copy Handler  Open source tool.  Handler is a free software for Windows systems, which allows you to take full control over the file copy and move operations. Program is highly customizable and fully integrates with the operating system. In addition, Copy Handler offers many advanced options such as task queuing, filtering files according to specific criteria, pausing and resuming copying files operations, changing the copying parameters on the fly and in many cases also offers a significant acceleration of the entire copying files process.
     
  2. SuperCopier Open source software released under GNU license. SuperCopier replaces windows explorer file copy and adds many features, transfer resuming, copy speed control, copy speed computation, better copy progress display, copy list editable while copying, error log, copy list saving/loading,...
     
  3. FastCopy: Abandonware (last version is dated 2012)  This is command line took which automatically selects different methods according to whether Source and DestDir are in the same or different HDD. Diff HDD Reading and writing are processed respectively in parallel by separate threads. Same HDD Reading is processed until the big buffer fills. When the big buffer filled, writing is started and processed in bulk. Because Reading/Writing are processed with no OS cache at all, other applications don't easily become slow. It can achieve Reading/Writing performance that is close to device limit. Include/Exclude Filter (UNIX Wildcard style) can be specified.  It does not hog resources, because MFC is not used. (Designed using Win32 API and C Runtime only). You can modify this software, because all source code has been opened to the public in the BSD license.
  4. RichCopy, an Advanced Alternative to RoboCopy.  Abandonware from 2009. RichCopy is a free utility developed by Ken Tamaru of Microsoft. The tool was first developed in 2001 and has been updated regularly to keep pace with evolving needs. Trust me when I tell you, this is the answer to all your file copying needs. What you'll find most striking the first time you take RichCopy out for a spin is that it's a multithreaded copying tool. That means that rather than copying one file at a time in serial order, RichCopy can open multiple threads simultaneously, allowing many files to be copied in parallel and cutting the total time required to complete the operation several times over. You can also pause and resume file copy operations, so if you lose network connectivity at any point, you can just pick up where you left off. Surprisingly it copies and moves files even faster than RoboCopy tool and XCOPY command. Since the tool was being used by the developers, they made it pretty effective and stable .

Top Visited
Switchboard
Latest
Past week
Past month

NEWS CONTENTS

Old News ;-)

[Oct 19, 2017] What is a better file copy alternative than the Windows default - Stack Overflow

Oct 19, 2017 | stackoverflow.com

Michael Stum , Aug 4, 2008 at 15:28

How about good old Command-Line Xcopy? With S: being the source and T: the target:
xcopy /K /R /E /I /S /C /H /G /X /Y s:\*.* t:\

/K Copies attributes. Normal Xcopy will reset read-only attributes.

/R Overwrites read-only files.

/E Copies directories and subdirectories, including empty ones.

/I If destination does not exist and copying more than one file, assumes that destination must be a directory.

/S Copies directories and subdirectories except empty ones.

/C Continues copying even if errors occur.

/H Copies hidden and system files also.

/Y Suppresses prompting to confirm you want to overwrite an existing destination file.

/G Allows the copying of encrypted files to destination that does not support encryption.

/X Copies file audit settings (implies /O).

(Edit: Added /G and /X which are new since a few years)

Miles , Oct 15, 2008 at 16:55

this works alot better than the regular windows copy. We had to copy a document server from one partition to another one time. I think we tried the windows copy first (said it was gonna take 20something hours). When we tried X-copy, it took less than 10 (and kept the windows permissions intact). – Miles Oct 15 '08 at 16:55

Oliver Zendel , Mar 11, 2014 at 9:29

xcopy has a serious bug if the path of a single file while copying has more than 256 characters. xcopy will abort the whole operation with an 'insufficient memory' error -> don't use xcopy! As an alternative, this will do pretty much what xcopy does without the bugs:

robocopy c:\srcdir d:\dstdir /XJ /SL /B /E /S /R:1 /W:1 /NFL /NS /NC /MT > mylog.txt 2>&1

Oliver Zendel Mar 11 '14 at 9:29

huseyint , Aug 4, 2008 at 15:21

Use Robocopy (Robust File Copy) .

NOTE:

In Windows Vista and Server 2008 when you type:

xcopy /?

you get:

NOTE: Xcopy is now deprecated, please use Robocopy.

So start getting used to robocopy :)

Gordon Bell , Oct 3, 2008 at 5:40

The primary complaint I've always had with XCOPY is that it always re-copies the files even if the destination file already exists with the same file size and modification date. – Gordon Bell Oct 3 '08 at 5:40

Wikipedia - robocopy

I find RoboCopy is a good alternative to xcopy. It supports high latency connections much better and supports resuming a copy.

References

Wikipedia - robocopy

Downloads

Edit Robocopy was introduced as a standard feature of Windows Vista and Windows Server 2008.

•Robocopy is shipped as part of the Windows Server 2003 resource kit and can be download from the Microsoft download site.


•A very simple GUI has also been release for RoboCopy on technet http://technet.microsoft.com/en-us/magazine/cc160891.aspx

Xcopy

Copies files and directories, including subdirectories.Syntax

xcopy Source [Destination] [/w] [/p] [/c] [/v] [/q] [/f] [/l] [/g] [/d[:mm-dd-yyyy]] [/u] [/i] [/s [/e]] [/t] [/k] [/r] [/h] [{/a|/m}] [/n] [/o] [/x] [/exclude:file1[+[file2]][+[file3]] [{/y|/-y}] [/z]

Top of page

Parameters

Source : Required. Specifies the location and names of the files you want to copy. This parameter must include either a drive or a path.

Destination : Specifies the destination of the files you want to copy. This parameter can include a drive letter and colon, a directory name, a file name, or a combination of these.

/w : Displays the following message and waits for your response before starting to copy files:

Press any key to begin copying file(s)

/p : Prompts you to confirm whether you want to create each destination file.

/c : Ignores errors.

/v : Verifies each file as it is written to the destination file to make sure that the destination files are identical to the source files.

/q : Suppresses the display of xcopy messages.

/f : Displays source and destination file names while copying.

/l : Displays a list of files that are to be copied.

/g : Creates decrypted destination files.

/d [ : mm-dd-yyyy ] : Copies source files changed on or after the specified date only. If you do not include a mm-dd-yyyy value, xcopy copies all Source files that are newer than existing Destination files. This command-line option allows you to update files that have changed.

/u : Copies files from Source that exist on Destination only.

/i : If Source is a directory or contains wildcards and Destination does not exist, xcopy assumes destination specifies a directory name and creates a new directory. Then, xcopy copies all specified files into the new directory. By default, xcopy prompts you to specify whether Destination is a file or a directory.

/s : Copies directories and subdirectories, unless they are empty. If you omit /s, xcopy works within a single directory.

/e : Copies all subdirectories, even if they are empty. Use /e with the /s and /t command-line options.

/t : Copies the subdirectory structure (that is, the tree) only, not files. To copy empty directories, you must include the /e command-line option.

/k : Copies files and retains the read-only attribute on destination files if present on the source files. By default, xcopy removes the read-only attribute.

/r : Copies read-only files.

/h : Copies files with hidden and system file attributes. By default, xcopy does not copy hidden or system files.

/a : Copies only source files that have their archive file attributes set. /a does not modify the archive file attribute of the source file. For information about how to set the archive file attribute by using attrib, see Related Topics.

/m : Copies source files that have their archive file attributes set. Unlike /a, /m turns off archive file attributes in the files that are specified in the source. For information about how to set the archive file attribute by using attrib, see Related Topics.

/n : Creates copies by using the NTFS short file or directory names. /n is required when you copy files or directories from an NTFS volume to a FAT volume or when the FAT file system naming convention (that is, 8.3 characters) is required on the destination file system. The destination file system can be FAT or NTFS.

/o : Copies file ownership and discretionary access control list (DACL) information.

/x : Copies file audit settings and system access control list (SACL) information (implies /o).

/exclude: filename1 [ + [ filename2 ]][ + [ filename3 ]] : Specifies a list of files containing strings.

/y : Suppresses prompting to confirm that you want to overwrite an existing destination file.

/-y : Prompts to confirm that you want to overwrite an existing destination file.

/z : Copies over a network in restartable mode.

/? : Displays help at the command prompt.

Top of page

Remarks

Top of page

Examples

To copy all the files and subdirectories (including any empty subdirectories) from drive A to drive B, type:

xcopy a: b: /s /e

To include any system or hidden files in the previous example, add the/h command-line option as follows:

xcopy a: b: /s /e /h

To update files in the \Reports directory with the files in the \Rawdata directory that have changed since December 29, 1993, type:

xcopy \rawdata \reports /d:12-29-1993

To update all the files that exist in \Reports in the previous example, regardless of date, type:

xcopy \rawdata \reports /u

To obtain a list of the files to be copied by the previous command (that is, without actually copying the files), type:

xcopy \rawdata \reports /d:12-29-1993 /l > xcopy.out

The file Xcopy.out lists every file that is to be copied.

To copy the \Customer directory and all subdirectories to the directory \\Public\Address on network drive H:, retain the read-only attribute, and be prompted when a new file is created on H:, type:

xcopy \customer h:\public\address /s /e /k /p

To issue the previous command, ensure that xcopy creates the \Address directory if it does not exist, and suppress the message that appears when you create a new directory, add the /i command-line option as follows:

xcopy \customer h:\public\address /s /e /k /p /i

You can create a batch program to perform xcopy operations and use the batch if command to process the exit code if an error occurs. For example, the following batch program uses replaceable parameters for the xcopy source and destination parameters:

@echo off 
rem COPYIT.BAT transfers all files in all subdirectories of
rem the source drive or directory (%1) to the destination rem drive or directory (%2) 
xcopy %1 %2 /s /e 
if errorlevel 4 goto lowmemory 
if errorlevel 2 goto abort 
if errorlevel 0 goto exit 
:lowmemory 
echo Insufficient memory to copy files or 
echo invalid drive or command-line syntax. 
goto exit 
:abort 
echo You pressed CTRL+C to end the copy operation. 
goto exit 
:exit 

To use this batch program to copy all files in the C:\Prgmcode directory and its subdirectories to drive B, type:

copyit c:\prgmcode b:

The command interpreter substitutes C:\Prgmcode for %1 and B: for %2, then uses xcopy with the /e and /s command-line options. If xcopy encounters an error, the batch program reads the exit code and goes to the label indicated in the appropriate IF ERRORLEVEL statement, then displays the appropriate message and exits from the batch program.

Alternatives to XCopy for copying lots of files

Stack Overflow
/D may be what you are looking for. I find it works quite fast for backing-up as existing files are not copied.
xcopy "O:\*.*" N:\Whatever /C /D /S /H 

/C Continues copying even if errors occur. 
/D:m-d-y Copies files changed on or after the specified date. 
    If no date is given, copies only those files whose source time 
    is newer than the destination time. 
/S Copies directories and subdirectories except empty ones. 
/H Copies hidden and system files also.

TeraCopy Home

Feb 16, 2010 | PCWorld

When last I visited TeraCopy (the Pro version) I loved its features, but I discovered that version was not 100% stable. For a task as critical as copying and moving files, there's no substitute for rock-solid. Happily, I had no problems with this more recent free Home version of the program and can honestly say that TeraCopy seems ready for prime time.

TeraCopy inserts itself in place of the Windows file handling functions, adding to the usual right-click option. TeraCopy's pause function alone is worth the download if you regularly copy large amounts of files and need to stop the process to perform another disk-intensive task. Windows only lets you cancel--not pause--which makes it difficult to ascertain exactly where you left off. You may also choose up front whether you want to overwrite all files, older ones only, skip files that already exist, or rename the file being copied to avoid overwriting the existing files. TeraCopy seemed a tad faster than Windows in my tests with a large mix of small and large files though a little slower than the Windows native routine when copying a single very large file.

TeraCopy provides a lot more information about the files being copied, such as their size and original location. It may be invoked as a standalone so you can select files from diverse locations .

Best Free File Copy Utility

Introduction

Hard drives are becoming larger with each advance made in the hardware industry. With the increase in storage size available the amount of files the average user keeps on their computer has also greatly expanded. But when you want to backup those files safely to either another partition or an external hard drive you need a copy utility that will not run into problems. I used my own music collection at 21.7GB and used several free copy utility programs to see which of them was the fastest, most reliable, and easy to use.

  1. FastCopy is my choice for the best file copy manager. When it comes to stability and speed, FastCopy gave me consistent results and beat every other manager in the running by several seconds. I also noted that during the transfer, even with the 64MB cache that it used, I was still not seeing a significant jump in resource usage. My RAM usage stayed consistently low throughout the entire transfer. The program is excellent which made it my top choice of the file managers I tested.

    It also includes shell integration with the program and x64 capabilities. The program can also be run without being installed which is a very nice touch if you only need it for certain tasks. It also includes an installer for shell integration and for use as your default copier. It will not override your Windows file copier even when installed.

    There are a lot of great things about this program but I cannot give this program a 10/10 for the following reasons. First, the uninstalling utility would probably be confusing for the average user since you have to enter the C:\Program Files\FastCopy folder in order to run setup again and choose to uninstall the program. With that said it did not show up in the "Uninstall Program" section of Windows 7 Control Panel which would be the optimal place for the program to be uninstalled. Secondly, shell integration must be configured via the "Options" menu and in order to choose what FastCopy options you would like added to the right-click menu since it does not automatically override your Windows copier.

    These to me are minor gripes as the shell extensions added manually are excellent and you can even choose to hide the transfer window in the system tray and hover your mouse over to see the transfer speeds. Also, built into the program is a file wiping utility that performs up to three passes over the file it is deleting for those wanting secure deletion of their files. A big thank you to the commenters for bringing this program, and its benchmarks, to my attention.

  2. UltraCopier has some pretty exciting features including an excellent file resume option, which allows for resuming of a file transfer from a disconnected USB key. So, for those of you playing at home, if you're in the middle of a file transfer from a USB key and it somehow becomes disconnected, when you plug it in again all you have to do is to hit resume and it will pick up where you left off! A great innovation, especially if you have a finicky Windows install that randomly disconnects your drive.

    It has also other useful options such as speed limitation and searching through the copy list. And as far as looks go, UltraCopier is a much more visually attractive copier replacement. It's open source, runs on multi-platforms including Windows, Mac and Linux.

    All of the software I reviewed here is excellent and it is up to the user to try these out and decide which one they prefer. For now, I'll be using FastCopy for large transfers and UltraCopier for daily transfers side by side.

  3. The next program that I tested was Unstoppable Copier. This one wins hands down in the reliability category. Not only does it transfer faster than the normal Windows copier, it is also able to recover partial files from damaged media or corrupt areas of a hard disk drive. This program is not a miracle worker but I gave it several scratched DVDs that were denied by my Windows copier and it was able to get me a complete backup of the files on the disk. I am very impressed by the software and its uses.

    I do not believe it is the intention of the project to do so, but I think this could be an even better program if it was able to be used in place of the Windows copier. It is portable and installable but both versions must be launched in order to do file transfers. The output of the program is outstanding and gives you a very good idea of which files have copied successfully and which files will be re-attempted. With its unique ability to recover damaged files, this copying program is the most reliable of the programs I tested.

  4. I recently used the 2.22 version of TeraCopy. I've seen a lot of complaints about stability with this program and so I decided to push it to the limits. Full on 21.7GB transfer of small files, pausing and resuming of said files, follow-up verification of the total transfer, and killing and re-starting of the same transfer. I will say that these features worked as intended on my setup which is Windows 7 x64 Home Premium.

    While this program is a beta, the features I tested performed as they should on my setup. My results are as individual as they come because obviously I cannot account for every variation but the method I used involved TeraCopy as the only program running. Even when I wasn't 'harassing' it by pausing and resuming, my setup is RAID 0, I tried copying small files and one ridiculously large file several times. Each time it did exactly what I asked it to do. The file verification also reassured me that everything transferred correctly even if I did manually verify later.

    I personally switch file copiers because I want something faster and more reliable than the Windows copier. In this respect as a Windows copier replacement TeraCopy still shines. I will have some people I know test this program and see if they run into any problems that perhaps I could not. But I still have to say that this is the most actively developed and impressively integrated file copier out right now.

    The fact that they have a freeware version that works with Windows 8 x64 (seriously, look at their page) and smooth total integration still makes this one of the best file copiers I've used to date. Which is why it remains a contender for the top spot. If a new release ends up equalling FastCopy on speed, resource usage and stability, then you all will be seeing a new top spot recommendation from me.

Other File Copiers

Besides the recommended file copy utilities, I'd also mention SuperCopier and KillCopy since both are stable and reliable. Both are faster than the built-in Windows copier. They run from the system tray and are light on resources. KillCopy is quite a bit faster on transfers and has a plethora of options for the advanced user. It gets a hard time for its looks but it has many skins available on its webpage if you do decide to go that route. None that make it as attractive as UltraCopier, in my opinion, but it does help.

Microsoft Robocopy

Microsoft's Robocopy, a command line directory replication command, gradually replaced the older xcopy. It rocks a whole host of features that make is the logical choice for transferring a large number of files. It's multi-threaded, has a ton of options, and generally outperforms vanilla Windows copy operations. Best of all, it's built right in to Redmond's operating system. Especially useful for network copy operations and backups, Robocopy doesn't stop to ask you one hundred questions while it copies over your music collection, either.

The reality of benchmarking file copy performance is that you need something fast to copy from and something fast to copy to. This is most important with SSDs. It doesn't matter if your drive can write sequentially at 500 MB/s if the source files are hosted on a USB 2.0-attached external hard drive. We're copying our test files from an Intel SSD DC S3700 to the drives in the chart below, taking source speed out of the equation.

There are 9065 files comprising the 16.2 GB payload. Some of the files are huge (up to 2 GB), while others are best described as tiny. On average, that's around 1.8 MB per file. The files are a mix of music, program, pictures, and random file types.

It's fair to say that this chart would look much different if we were copying from a hard drive to a SSD. Even if the disk drive's sequential throughput wasn't a bottleneck, it'd still choke on the smaller files.

Recommended Links

Google matched content

Softpanorama Recommended

Top articles

Sites

List of file copying software - Wikipedia, the free encyclopedia

TeraCopy for Microsoft Windows - Code Sector

TeraCopy - Wikipedia, the free encyclopedia

TeraCopy - Wikipedia, the free encyclopedia

FastCopy 2.11 Freeware Download - Freeware Files.com - Utilities Category

5 Free Bulk File Copy Software to Copy Files Fast Free Software