cp command

The command very few sysadmin understand in full, and even fewer use its vital capabilities despite daily use

The cp command in Unix can copy one or several files. While it sound simple that command actually is pretty complex, because it does simple thing in a very complex environment. BSD has also cpdup command which allows to mirror filesystems. Some Linux distributes ported it. For the rest Rsync and tar are good enough alternatives.

The key problem with cp utility is that it by default it has a non-intuitive behaviour for Windows users: it does not preserve timestamp of the file. You need to use option -p to force this behaviour.

-p  -- Preserve the characteristics of the source_file. Copy the contents, modification times, and permission modes of the source_file to the destination files.

Like most modern Linux utilities from GNU toolbox it has way too many options. I think it has more then 30 options. Among other things it can create symbolic link to files in some other directory.  Few sysadmin  fully understand capabilities of cp command and usually use a very limited subset of its functionality. That's sad but that the net result of overcomplexity of Unix/Linux.

Among options  implemented in GNU version of cp that almost nobody know and use we can mention

• -n, --no-clobber  Do not overwrite file if it exists.
• -L, --dereference

  always follow symbolic links in SOURCE

• -b --backup[=CONTROL]  make a backup of each existing destination. The backup suffix is '~', unless set with --suffix or SIMPLE_BACKUP_SUFFIX. The version control method may be selected via the --backup option or through the VERSION_CONTROL environment variable
• -l, --link

  hard link files instead of copying

• -s, --symbolic-link

  make symbolic links instead of copying

• -v, --verbose

  explain what is being done

• -x, --one-file-system

  stay on this file system

For each file cp reads the contents of a file and creates a new file, or overwrites an existing file. If can also recursively copy subtrees of the Unix directory tree.

The target is usually a directory unless both source and target arguments are files or (badly documented or may be  redundant ) option -T option is specified ( -T, --no-target-directory treats DEST as a normal file )

There are several formats of the cp command that allow you to:

• Copy one file to another
• Copy multiple files to a directory
• Copy input from your keyboard to a file
• Copy a file to your terminal

Following is the general format of the cp command.

cp    [ -ip ]  source_file destination_file
cp    [ -ipr ] source_file_list destination_directory
cp -r [ -ip ]  source_directory destination_directory

The following list describes typical three options that may be used to control how cp functions.

• -i  -- Interactive confirmation is required. You are prompted if the copy will overwrite an existing file. If your answer contains a y, the copy is performed. Otherwise, the copy is not performed. 
• -p  -- Preserve the characteristics of the source_file. Copy the contents, modification times, and permission modes of the source_file to the destination files.
• -r   -- Recursively copy any source directories. If a directory is given as the source file, then all of its files and subdirectories are copied. The destination must be a directory. 

The following list describes the arguments that may be passed to the cp command.

1. The first format copies one ordinary file to a new file. The source_file is the pathname of the file being copied. The destination_file is the pathname of the file being copied to by cp.
2. The second format will copy one or more files to a specific directory. The source_file_list is a list of pathnames of the files being copied. If you specify a directory with the -r option, the entire directory tree is copied to the destination directory.
3. The third format allows you to copy an entire directory structure to a new directory.

If the destination_file  exists and you have write permissions, cp will overwrite the contents of the file. The access permissions and ownership of the overwritten file will be used for the new destination_file.

If the destination_file  does not exist, the access permissions of the source_file  are used. Your ownership and group IDs are used for the new file if you are the user performing the cp command.

You cannot copy a file to itself.

Some common formats are:

   cp letter letter.bak    # make a backup copy of letter
   cp letter ../tmp        # copy letter to temporary working directory
   cp letter ../LTRS       # copy letter to original location (mv is better)
   cp /dev/tty note        # copy input from keyboard to file "note"

For copying directories you can also use rsync

It's also possible by using rsync, for example:

rsync -vua src/ dst/

NEWS CONTENTS

• 20200323 : Copy Specific File Types While Keeping Directory Structure In Linux by sk ( Mar 19, 2020 , www.ostechnix.com )
• 20190212* 16 Useful 'cp' Command Examples for Linux Beginners ( Feb 12, 2019 , www.linuxtechi.com ) [Recommended]
• 20170505 : As Unix does not have a rename command usage of mv for renaming can lead to SNAFU ( www.softpanorama.org )
• 20170505 : The key problem with cp utility is that it does not preserve timestamp of the file. ( www.vanityfair.com )
• 20170505 : BigAdmin Submitted Article How to Safely Move and Copy Files and Directories by Darek Licznerski. This article discusses the problems that can occur when you move and copy files and directories, and the author also offers solutions. ( BigAdmin Submitted Article How to Safely Move and Copy Files and Directories, )
• 20060726 : BigAdmin Submitted Article How to Safely Move and Copy Files and Directories ( BigAdmin Submitted Article How to Safely Move and Copy Files and Directories, Jul 26, 2006 )

Old News ;-)

[Mar 23, 2020] Copy Specific File Types While Keeping Directory Structure In Linux by sk

I think this approach is way too complex. A simpler and more reliable approach is first to create directory structure and then as the second statge to copy files.

Use of cp command option --parents -- create the intermediate parent directories if needed to preserve the parent directory structure. is interesting though

Notable quotes:

"... create the intermediate parent directories if needed to preserve the parent directory structure. ..."

Mar 19, 2020 | www.ostechnix.com

Copy Specific File Types While Keeping Directory Structure In Linux

by sk · March 19, 2020

A while ago, we learned how to copy certain type of files from one directory to another in Linux. Today we are going to do the same but preserve the directory structure as well. This brief tutorial explains how to copy specific file types while keeping the directory structure in Linux. Here I have given two different ways to do this. Just pick any one that works for you. Copy Specific File Types While Keeping Directory Structure In Linux

Picture this scenario.

I have a directory named "Linux" with different type of files saved in different sub-directories. Have a look at the following directory structure:

$ tree Linux/
Linux/
├── dir1
│   ├── English
│   │   └── Kina - Can We Kiss Forever.mp3
│   ├── Instrumental
│   │   └── Chill Study Beats.mp3
│   └── Tamil
│       ├── Kannan Vanthu.mp3
│       └── yarenna.mp3
├── dir2
│   ├── file.docx
│   └── Raja Raja Chozan Naan.mp3
├── dir3
│   ├── Bamboo Flute - Meditation - Healing - Sleep - Zen.mp3
│   └── pic.jpg
└── dir4
    ├── Aaruyirae.mp3
    └── video.mp4

7 directories, 10 files

As you see in the above directory structure, the Linux directory has four sub-directories namely dir1 , dir2 , dir3 and dir4 . The mp3 files are scattered across all four sub-directories. Now, I want to copy all mp3 files to another directory named "ostechnix" and also I want to keep the same directory structure in the target directory.

First we will see how to do this using "find" command.

Method 1 -- Copy specific file types while preserving directory structure using "find" and "cp" or "cpio" commands

Go to the source directory:

$ cd Linux/

And copy all mp3 fie types using "find" command:

$ find . -name '*.mp3' -exec cp --parents \{\} ~/ostechnix \;

Let us break down the above command and see what each option does.

• find -- command to find files and folders in Unix-like systems.
• the dot (.) -- represents we copy the contents from current directory.
• -name '*.mp3' -- search for files matching with extension .mp3.
• -exec cp -- execute the 'cp' command to copy files from source to destination directory.
• --parents -- create the intermediate parent directories if needed to preserve the parent directory structure.
• \{\} -- is automatically replaced with the file name of the files found by 'find' command. And the braces are escaped to protect them from expansion by the shell in some "find" command versions. You can also use {} without escape characters.
• ~/ostechnix -- target directory to save the matching files.
• \; -- indicates it that the commands to be executed are now complete, and to carry out the command again on the next match.

This command will find and copy all mp3 type files from ~/Linux directory to ~/ostechnix directory. And also it preserves the same directory structure in the target directory.

You can verify it using "tree" command at both locations like below.

As you see in the above output, the destination directory only has the mp3 files and its directory structure is same as source directory.

If you are doing this from some other location, specify the full path of source directory like below.

$ find ~/Linux -name '*.mp3' -exec cp --parents \{\} ~/ostechnix \;

This command will find all of the file in Linux/<sub-directories> location and copy them to ostechnix/~/Linux/<sub-directories> .

If --parents option doesn't work, you can combine find command with cpio command to copy files keeping directory structure.

$ find . -name '*.mp3' | cpio -pdm  ~/ostechnix

Here,

• cpio -- Command to copy files to and from archives.
• -p -- Read a list of file names from the standard input and copy them to the specified directory.
• -d -- Create directories where needed.
• -m -- Preserve file modification time.

For more details, refer man pages.

$ man find
$ man cp
$ man cpio

---

Suggested read:

• How To Automatically Organize Similar Type Of Files In Specific Folders

---

Method 2 -- Copy specific file types while preserving directory structure using Rsync

Rsync is a powerful tool used to/from local and remote systems. To copy certain type of files from one directory to another while keeping the parent directory structure, run:

$ rsync -a -m --include '*/' --include '*.mp3' --exclude '*' ~/Linux/ ~/ostechnix

Or,

$ rsync -a --prune-empty-dirs --include '*/' --include '*.mp3' --exclude '*' ~/Linux/ ~/ostechnix

Here,

• rsync -- remote (and local) file-copying tool.
• -a -- archive mode to preserve almost everything (including symlinks, modification dates, file permissions, owners etc.)
• -m, --prune-empty-dirs -- prune empty directories from source tree. If you want to include empty directories, just remove this option from the above command.
• --include="*/" --include="*.mp3″ --exclude="*" -- To include only specific files, you first need to include those specific files, then exclude all other files. In our case, we have included *.mp3 files and exclude everything else.
• ~/Linux -- Source directory.
• ~/ostechnix -- Destination directory.

Sample output:

For more details, refer man pages.

$ man rsync

Hope this helps.

Thanks for stopping by!

[Feb 12, 2019] 16 Useful 'cp' Command Examples for Linux Beginners

Highly recommended!

Feb 12, 2019 | www.linuxtechi.com

Example:6) Archive files and directory during copy (-a)

While copying a directory using cp command we generally use -r or -R option, but in place of -r option we can use '-a' which will archive the files and directory during copy, example is shown below,

root@linuxtechi:~# cp -a /home/linuxtechi /mnt/backup/
root@linuxtechi:~# ls -l /mnt/backup/linuxtechi/
total 24
drwxr-xr-x 2 root root 4096 Feb  3 17:41 data
-rw-r--r-- 1 root root    7 Feb  3 17:39 file_1.txt
-rw-r--r-- 1 root root    7 Feb  3 17:39 file_2.txt
-rw-r--r-- 1 root root    7 Feb  3 17:39 file_3.txt
-rw-r--r-- 1 root root    7 Feb  3 17:39 file_4.txt
-rw-r--r-- 1 root root    7 Feb  3 17:40 file_5txt
-rw-r--r-- 1 root root    0 Feb  3 17:39 file_5.txt
root@linuxtechi:~#

Example:7) Copy only when source file is newer than the target file (-u)

There can be some scenarios where you want copy the files only if the source files are newer than the destination ones. This can be easily achieved using " -u " option in the cp command.

In the Example:6 we have copied the linuxtechi home directory to /mnt/backup folder, in the linuxtechi home folder we have 5 txt files, let's edit couple of them and then copy all the txt files using "cp -u".

root@linuxtechi:~# cd /home/linuxtechi/
root@linuxtechi:/home/linuxtechi# echo "LinuxRocks" >> file_1.txt
root@linuxtechi:/home/linuxtechi# echo "LinuxRocks" >> file_4.txt
root@linuxtechi:/home/linuxtechi# cp -v -u  file_*.txt /mnt/backup/linuxtechi/
'file_1.txt' -> '/mnt/backup/linuxtechi/file_1.txt'
'file_4.txt' -> '/mnt/backup/linuxtechi/file_4.txt'
root@linuxtechi:/home/linuxtechi#

Example:8) Do not overwrite the existing file while copying (-n)

There are some scenarios where you don't want to overwrite the existing destination files while copying. This can be accomplished using the option '-n' in 'cp' command

root@linuxtechi:~# cp -i /etc/passwd /mnt/backup/
cp: overwrite '/mnt/backup/passwd'?

As you can see in above command, it is prompting us to overwrite the existing file, if you use -n then it will not prompt for the overwrite and also will not overwrite the existing file.

root@linuxtechi:~# cp -n /etc/passwd /mnt/backup/
root@linuxtechi:~#

Example:9) Creating symbolic links using cp command (-s)

Let's assume we want to create symbolic link of a file instead copying using cp command, for such scenarios use '-s' option in cp command, example is shown below

root@linuxtechi:~# cp -s /home/linuxtechi/file_1.txt /mnt/backup/
root@linuxtechi:~# cd /mnt/backup/
root@linuxtechi:/mnt/backup# ls -l file_1.txt
lrwxrwxrwx 1 root root 27 Feb  5 18:37 file_1.txt -> /home/linuxtechi/file_1.txt
root@linuxtechi:/mnt/backup#

Example:10) Creating Hard link using cp command (-l)

If you want to create hard link of a file instead copy using cp command, then use '-l' option. example is shown below,

root@linuxtechi:~# cp -l /home/linuxtechi/devops.txt /mnt/backup/
root@linuxtechi:~#

As we know in hard link, source and linked file will have the same inode numbers, let's verify this using following commands,

root@linuxtechi:~# ls -li /mnt/backup/devops.txt
918196 -rw-r--r-- 2 root root 37 Feb  5 20:02 /mnt/backup/devops.txt
root@linuxtechi:~# ls -li /home/linuxtechi/devops.txt
918196 -rw-r--r-- 2 root root 37 Feb  5 20:02 /home/linuxtechi/devops.txt
root@linuxtechi:

Example:11) Copying attributes from source to destination (–attributes-only)

If you want to copy only the attributes from source to destination using cp command, then use option " –attributes-only "

root@linuxtechi:/home/linuxtechi# cp --attributes-only /home/linuxtechi/distributions.txt /mnt/backup/
root@linuxtechi:/home/linuxtechi# ls -l /home/linuxtechi/distributions.txt
-rw-r--r-- 1 root root 41 Feb  5 19:31 /home/linuxtechi/distributions.txt
root@linuxtechi:/home/linuxtechi# ls -l /mnt/backup/distributions.txt
-rw-r--r-- 1 root root 0 Feb  5 19:34 /mnt/backup/distributions.txt
root@linuxtechi:/home/linuxtechi#

In the above command, we have copied the distribution.txt file from linuxtechi home directory to /mnt/backup folder, if you have noticed, only the attributes are copied, and content is skipped. Size of distribution.txt under /mn/backup folder is zero bytes.

Example:12) Creating backup of existing destination file while copying (–backup)

Default behavior of cp command is to overwrite the file on destination if the same file exists, if you want to make a backup of existing destination file during the copy operation then use ' –backup ' option, example is shown below,

root@linuxtechi:~# cp --backup=simple -v /home/linuxtechi/distributions.txt /mnt/backup/distributions.txt
'/home/linuxtechi/distributions.txt' -> '/mnt/backup/distributions.txt' (backup: '/mnt/backup/distributions.txt~')
root@linuxtechi:~#

If you have noticed, backup has been created and appended tilde symbol at end of file. backup option accept following parameters

• none, off – never make backups
• numbered, t – make numbered backups
• existing, nil – numbered if numbered backups exist, simple otherwise
• simple, never – always make simple backups

Example:13) Preserve mode, ownership and timestamps while copying (-p)

If you want to preserve the file attributes like mode, ownership and timestamps while copying then use -p option in cp command, example is demonstrated below,

root@linuxtechi:~# cd /home/linuxtechi/
root@linuxtechi:/home/linuxtechi# cp -p devops.txt /mnt/backup/
root@linuxtechi:/home/linuxtechi# ls -l devops.txt
-rw-r--r-- 1 root root 37 Feb  5 20:02 devops.txt
root@linuxtechi:/home/linuxtechi# ls -l /mnt/backup/devops.txt
-rw-r--r-- 1 root root 37 Feb  5 20:02 /mnt/backup/devops.txt
root@linuxtechi:/home/linuxtechi#

Example:14) Do not follow symbolic links in Source while copying (-P)

If you do not want to follow the symbolic links of source while copying then use -P option in cp command, example is shown below

root@linuxtechi:~# cd /home/linuxtechi/
root@linuxtechi:/home/linuxtechi# ls -l /opt/nix-release.txt
lrwxrwxrwx 1 root root 14 Feb  9 12:28 /opt/nix-release.txt -> os-release.txt
root@linuxtechi:/home/linuxtechi#
root@linuxtechi:/home/linuxtechi# cp -P os-release.txt /mnt/backup/
root@linuxtechi:/home/linuxtechi# ls -l /mnt/backup/os-release.txt
-rw-r--r-- 1 root root 35 Feb  9 12:29 /mnt/backup/os-release.txt
root@linuxtechi:/home/linuxtechi#

Note: Default behavior of cp command is to follow the symbolic links in source while copying.

Example:15) Copy the files and directory forcefully using -f option

There can be some scenarios where existing destination file cannot be opened and removed. And if you have healthy file which can be copied in place of existing destination file, then use cp command along with -f option

root@linuxtechi:/home/linuxtechi# cp -f distributions.txt  /mnt/backup/
root@linuxtechi:/home/linuxtechi#

Example:16) Copy sparse files using sparse option in cp command

Sparse is a regular file which contains long sequence of zero bytes that doesn't consume any physical disk block. One of benefit of sparse file is that it does not consume much disk space and read operation on that file would be quite fast.

Let's assume we have sparse cloud image named as "ubuntu-cloud.img"

root@linuxtechi:/home/linuxtechi# du -sh ubuntu-cloud.img
12M     ubuntu-cloud.img
root@linuxtechi:/home/linuxtechi# cp --sparse=always ubuntu-cloud.img /mnt/backup/
root@linuxtechi:/home/linuxtechi# du -sh /mnt/backup/ubuntu-cloud.img
0       /mnt/backup/ubuntu-cloud.img
root@linuxtechi:/home/linuxtechi#

Different options can be used while using sparse parameter in cp command,

• sparse=auto
• sparse-always
• sparse=never

That's all from this article, I hope it helps you to understand the cp command more effectively. Please do share your feedback and comments

[May 05, 2017] As Unix does not have a rename command usage of mv for renaming can lead to SNAFU

www.softpanorama.org

If destination does not exist it behaves as rename command but if destination exists and is directory it move it one level up

For example, if you have directories /home and home2 and want to move all subdirectories from /home2 to /home and the directory /home is empty you can't use

mv home2 home

if you forget to remove the directory /home, mv silently will create /home/home2 directory and you have a problem if this is user home directories.

[May 05, 2017] The key problem with cp utility is that it does not preserve timestamp of the file.

Expected behaviour of copy command by windows users is that it preserves attributes. But this in not true for Unix cp command.

Using -r option without -p option destroys all timestamps.

www.vanityfair.com

-p -- Preserve the characteristics of the source_file. Copy the contents, modification times, and permission modes of the source_file to the destination files.

You might wish to create an alias

alias cp='cp -p'

as I can't imagine case where regular Unix behaviour is desirable.

BigAdmin Submitted Article How to Safely Move and Copy Files and Directories by Darek Licznerski. This article discusses the problems that can occur when you move and copy files and directories, and the author also offers solutions.

Contents:

• Introduction
• Caveats
  • NFS
  • Symbolic Links
  • Unknown Destination
  • Problems With Access
  • Interruptions During the Copying or Moving Process
  • Moving Between Different Devices
  • Owner, Groups, Permissions
• Summary and Some Issues

Introduction

You could say, "This is not a problem for me because I can copy files with the cp or cp -r commands or move files with the mv command." However, sometimes you can break something and lose data.

I recommend you avoid such solutions as the cp or mv commands when you need to copy or move big files, large numbers of files, or large amounts of data. Also, avoid using commands such as cp or mv when you need to keep the same owner, group, permissions, and all the other information about the data.

Caveats

NFS

You can see problems when the destination or source directory is mounted on a local machine from a remote machine via NFS or through the network with any other protocol. To avoid such problems check the source and destination directories via commands. For example:

# df -k <directory>
where <directory> - source or destination directory

Or use this for the Solaris 9 Operating System (OS) and later versions:

# df -h <directory>
where <directory> - source or destination directory

Example:

# df -k  /source_directory
Filesystem      kbytes    used     avail      capacity    Mounted on
127.0.0.1:/source
                34750583  28174655 6228423    82%     /source_directory

Here the directory /source from the host with the IP address 127.0.0.1 (in this case localhost) is mounted on a local machine under the directory /source_directory. To check what type of service is used to mount the directory, run a command such as the following:

# mount -v | grep "<directory>" where <directory> - source or destination directory

Symbolic Links

You can see problems when the destination or source directory -- or even worse, the subdirectories -- are linked in the source directory.

When you run the cp -r command on the directory that contains symbolic links, the links in the destination directory will be changed to the real data to which those links pointed. If you use the cp -r command directory to copy about 1 Gbyte of data that contains large quantities of links, the destination will be 1 Gbyte plus all the data to which those links pointed. Sometimes the result could be huge and could overload your system.

To check if the source directory contains symbolic links, do the following:

# cd <directory>
where <directory> - source or destination directory

# find . -type link | more

To count the number of links, use this:

# find . -type link | wc

Here is the result of copying directory example01 to the new one, example02, using the command cp -r ./example01 ./example02:

# find ./example01 -type link | wc
    2440    2440  171743

# du -ks ./example01
2772092 ./example01

# find ./example02 -type link | wc
       0       0       0 

# du -ks ./example02
6326172 ./example02

Unknown Destination

Problems can surface when you are not familiar with the real location, destination, or source directory because they may point somewhere else.

For example:

# df -k /source_directory
Filesystem         kbytes    used    avail     capacity    Mounted on
/dev/dsk/c0t0d0s0  34750583  28174655 6228423  82%   /source_directory   

# df -k /destination_directory
Filesystem    kbytes    used    avail     capacity    Mounted on
127.0.0.1:/destination
              34750583  28174655  6228423   82% /destination_directory

Here /source_directory is the directory created on a local disk (on the root partition). /destination_directory is the directory /destination on the host with the IP address 127.0.0.1 (in this case localhost), mounted on a local machine. For the purpose of this example, the IP address is localhost. However, in a real scenario, it could be an IP from a different subnet, and the packets (data) would be transferred through the network and who knows where.

When the source or destination directory is mounted through the network, you need to know the following:

• The real way to reach such a destination or source directory
• How long it takes to copy data to and from the directory
• Whether the data will be copied through any other machines or devices (like network accelerators)

The best way is to copy some example data to the destination and get a measurement.

Problems With Access

If you are unfamiliar with how to access a destination or source directory, you may be unable to access the directories or subdirectories.

When you decide to move data using the mv command, make sure that no processes are running in the source directory that will be moved.

Sometimes you can't access the mounted directory because you have the wrong permissions on the remote host for the directory you are trying to mount. Make sure that the destination directory is not mounted as a read-only file system. For example:

# df -k /destination_directory
Filesystem kbytes     used     avail    capacity    Mounted on
127.0.0.1:/destination
          34750583   28174655   6228423  82%     /destination_directory 

# mount -v | grep "<directory>"

where <directory> - destination directory

Also, check that no read-only flags are set up. The best way is to try to write an example file to the destination before copying or moving large amounts of data.

Interruptions During the Copying or Moving Process

Various problems are possible, for example, if you are using the mv command from the source to the destination directory and one of them is mounted through the network. If the network becomes unavailable while the files are being moved, or you kill the terminal or kill the data moving process by mistake, data will be moved partially and you also can lose data.

A problem also occurs if you run the cp command from the source to the destination directory, with one of them mounted through the network, and the network becomes unavailable while files are being copied. In this case data will be moved partially.

When you do such moving of data remotely and, for example, you export DISPLAY to your machine, there is a high probability that the process could be interrupted. So make sure that the place from which you execute such commands is stable.

To check the terminal name, do the following:

# tty
/dev/console

# tty
/dev/pts/1

If you are logged in to the console you also have control during the reboot. Consider using the nohup command to avoid interruptions caused by long-running processes.

Moving Between Different Devices

It takes much longer to move files and directories via the mv command among directories that are not created on the same partition or disks than it does to move directories sharing the same partition. (On the same partition, this mv command works at once, but between different devices you can be surprised by how long it takes.) The time required mostly depends on having access to such a partition or device. Make sure that during the moving of files the machine will not be rebooted and your moving process will not be interrupted.

For example:

# df -k /source_directory
Filesystem         kbytes    used      avail    capacity  Mounted on
/dev/dsk/c0t0d0s0  34750583  28174655  6228423  82%  /source_directory   

# df -k /destination_directory
Filesystem        kbytes    used     avail   capacity  Mounted on
/dev/md/dsk/d30   34750583  28174655 6228423 82% /destination_directory

In the previous case the /source_directory is the directory created on the local disk (root partition) and the /destination_directory is the directory created on the metadevice. So the time would not be the same as it would be on the same partition.

Make sure that the destination and source directory have the same block structure. If not, avoid using cp or mv commands. To check this you can create the same text file on each device and run the du -ks command on each file.

For example:

# du -ks ./example.txt
4       ./example.txt

# du -ks /tmp/example.txt
8       /tmp/example.txt

But the diff or sdiff -s commands show no differences.

Owner, Groups, Permissions

Make sure that owner, group, permissions, and any other file information are preserved. Some users or services might lose access to such data in the new location, and you would not be aware of that. Sometimes the service could be a critical part of the system.

Summary and Some Issues

Instead of using mv or cp commands to move or copy large amounts of data, consider the following procedure:

1. Make sure that none of the caveats mentioned previously would apply.
2. Verify disk spaces.

   For example:

   # df -k <directory>
   where <directory> - source directory

   # df -k <directory>
   where <directory> - destination directory

3. Copy data to the destination using commands such as cpio (recommended), tar, rsync, ufsdump, or ufsrestore.

   Example:

   Let the source directory be /source, and let the destination directory be /destination.

   # cd  /source
   # cd ..
   # find  ./source  -depth -print | cpio -cvo> /destination/source_data.cpio
   # cd /destination
   # cpio -icvmdI ./source_data.cpio
   # rm -rf ./source_data.cpio
   		

   This -c option is important in case the data is copied among different types of machines. This option provides read or write header information in ASCII character form for portability. There are no UID or GID restrictions associated with this header format. Use this option between SVR4-based machines, or use the -H odc option between unknown machines. The -c option implies the use of expanded device numbers, which are only supported on SVR4-based systems. Use the -H odc option when you are transferring files between the SunOS 4 or Interactive UNIX and the Solaris 2.6 OS or compatible versions.

   The procedure above first creates a file in the /destination directory that contains all the data packed via the cpio command and then unpacks this file in the destination.

   For more information, see the manual pages for the cpio command and other commands if needed.

   Also refer to:

   • System Administration Guide (for the version of your installed system)
   • GNU cpio
   
   
4. Compare data in the source directory and in the destination directory via commands such as sdiff -s, diff, du -ks, find, cmp, and dircmp.

   Example:

   Let the source directory be /source, and let the destination directory be /destination.

   # cd /source
   # cd ..
   # dircmp -s ./source ./destination
   		

5. If you intend to move data instead of copying, and you are completely sure that all data in the new location is correct, just remove all data in the source location or back it up somewhere else.

[Jul 26, 2006] BigAdmin Submitted Article How to Safely Move and Copy Files and Directories

Abstract: This article discusses the problems that can occur when you move and copy files and directories, and the author also offers solutions.

Contents:

• Introduction
• Caveats
  • NFS
  • Symbolic Links
  • Unknown Destination
  • Problems With Access
  • Interruptions During the Copying or Moving Process
  • Moving Between Different Devices
  • Owner, Groups, Permissions
• Summary and Some Issues

Introduction

You could say, "This is not a problem for me because I can copy files with the cp or cp -r commands or move files with the mv command." However, sometimes you can break something and lose data.

I recommend you avoid such solutions as the cp or mv commands when you need to copy or move big files, large numbers of files, or large amounts of data. Also, avoid using commands such as cp or mv when you need to keep the same owner, group, permissions, and all the other information about the data.

Recommended Links

Softpanorama Recommended

Top articles

[Feb 12, 2019] 16 Useful 'cp' Command Examples for Linux Beginners Published on Feb 12, 2019 | www.linuxtechi.com

Sites

Linux Command Directory cp

cp - Linux Command - Unix Command

Linux Guide-Getting Help - Wikibooks, collection of open-content textbooks

Reference

cp [OPTION]... [-T] SOURCE DEST
cp [OPTION]... SOURCE... DIRECTORY
cp [OPTION]... -t DIRECTORY SOURCE...

Copy SOURCE to DEST, or multiple SOURCE(s) to DIRECTORY.

Mandatory arguments to long options are mandatory for short options too.

-a, --archive

same as -dpR

--backup[=CONTROL]

make a backup of each existing destination file

-b

like --backup but does not accept an argument

--copy-contents

copy contents of special files when recursive

-d

same as --no-dereference --preserve=link

-f, --force

if an existing destination file cannot be opened, remove it and try again

-i, --interactive

prompt before overwrite

-H

follow command-line symbolic links

-l, --link

link files instead of copying

-L, --dereference

always follow symbolic links

-n, --no-clobber

do not overwrite an existing file (overrides a previous -i option)

-P, --no-dereference

never follow symbolic links

-p

same as --preserve=mode,ownership,timestamps

--preserve[=ATTR_LIST]

preserve the specified attributes (default: mode,ownership,timestamps) and security contexts, if possible additional attributes: links, all

--no-preserve=ATTR_LIST

don't preserve the specified attributes

--parents

use full source file name under DIRECTORY

-R, -r, --recursive

copy directories recursively

--remove-destination

remove each existing destination file before attempting to open it (contrast with --force)

--sparse=WHEN

control creation of sparse files

--strip-trailing-slashes remove any trailing slashes from each SOURCE

argument

-s, --symbolic-link

make symbolic links instead of copying

-S, --suffix=SUFFIX

override the usual backup suffix

-t, --target-directory=DIRECTORY

copy all SOURCE arguments into DIRECTORY

-T, --no-target-directory

treat DEST as a normal file

-u, --update

copy only when the SOURCE file is newer than the destination file or when the destination file is missing

-v, --verbose

explain what is being done

-x, --one-file-system

stay on this file system

--help

display this help and exit

-Z, --context=CONTEXT

set security context of copy to CONTEXT

--version

output version information and exit

By default, sparse SOURCE files are detected by a crude heuristic and the corresponding DEST file is made sparse as well. That is the behavior selected by --sparse=auto. Specify --sparse=always to create a sparse DEST file whenever the SOURCE file contains a long enough sequence of zero bytes. Use --sparse=never to inhibit creation of sparse files.

The backup suffix is '~', unless set with --suffix or SIMPLE_BACKUP_SUFFIX. The version control method may be selected via the --backup option or through the VERSION_CONTROL environment variable. Here are the values:

none, off

never make backups (even if --backup is given)

numbered, t

make numbered backups

existing, nil

numbered if numbered backups exist, simple otherwise

simple, never

always make simple backups

As a special case, cp makes a backup of SOURCE when the force and backup options are given and SOURCE and DEST are the same name for an existing, regular file.

docs.sun.com man pages section 1 User Commands

cp– copy files

/usr/bin/cp [-fip@] source_file target_file
/usr/bin/cp [-fip@] source_file… target
/usr/bin/cp -r| -R [-H| -L| -P] [-fip@] source_dir… target
/usr/xpg4/bin/cp [-fip@] source_file target_file
/usr/xpg4/bin/cp [-fip@] source_file… target
/usr/xpg4/bin/cp -r| -R [-H| -L| -P] [-fip@] source_dir… target


In the first synopsis form, neither source_file nor target_file are directory files, nor can they have the same name. The cp utility copies the contents of source_file to the destination path named by target_file. If target_file exists, cp overwrites its contents, but the mode (and ACL if applicable), owner, and group associated with it are not changed. The last modification time of target_file and the last access time of source_file are set to the time the copy was made.

If target_file does not exist, cp creates a new file named target_file that has the same mode as source_file except that the sticky bit is not set unless the user is super-user. In this case, the owner and group of target_file are those of the user, unless the setgid bit is set on the directory containing the newly created file.

If the directory's setgid bit is set, the newly created file has the group of the containing directory rather than of the creating user. If target_file is a link to another file, cp overwrites the link destination with the contents of source_file; the link(s) from target_file remains.

In the second synopsis form, one or more source_files are copied to the directory specified by target. It is an error if any source_file is a file of type directory, if target either does not exist or is not a directory.

In the third synopsis form, one or more directories specified by source_dir are copied to the directory specified by target. Either -r or -R must be specified. For each source_dir, cp copies all files and subdirectories.

OPTIONS

The following options are supported for both /usr/bin/cp and /usr/xpg4/bin/cp:

-f

Unlink. If a file descriptor for a destination file cannot be obtained, this option attempts to unlink the destination file and proceed.

-H

Takes actions based on the type and contents of the file referenced by any symbolic link specified as a source_file operand.

-i

Interactive. cp prompts for confirmation whenever the copy would overwrite an existing target. A y answer means that the copy should proceed. Any other answer prevents cp from overwriting target.

-L

Takes actions based on the type and contents of the file referenced by any symbolic link specified as a source_file operand or any symbolic links encountered during traversal of a file hierarchy.

-P

Takes actions on any symbolic link specified as a source_file operand or any symbolic link encountered during traversal of a file hierarchy.

-r

Recursive. cp copies the directory and all its files, including any subdirectories and their files to target. Unless the -H, -L, or -P option is specified, the -L option is used as the default mode.

-R

Same as -r, except pipes are replicated, not read from.

-@

Preserves extended attributes. cp attempts to copy all of the source file's extended attributes along with the file data to the destination file.

Specifying more than one of the mutually-exclusive options -H, -L, and -P is not considered an error. The last option specified determines the behavior of the utility.

/usr/bin/cp

The following option is supported for /usr/bin/cp only:

-p

Preserve. cp duplicates not only the contents of source_file, but also preserves the owner and group id, permission modes, modification and access time, ACLs, and extended attributes, if applicable. The command can fail if ACLs are copied to a file system without appropriate support. The command does not fail if unable to preserve extended attributes, modification and access time, or permission modes. If unable to preserve owner and group id, cp does not fail, and it clearsS_ISUID and S_ISGID bits in the target. cp prints a diagnostic message to stderr and return a non-zero exit status if unable to clear these bits.

In order to preserve the owner and group id, permission modes, and modification and access times, users must have the appropriate file access permissions. This includes being superuser or the same owner id as the destination file.

When both -p and -@ options are specified, the -p option determines the behavior. However, the command can fail if unable to preserve extended attributes.

/usr/xpg4/bin/cp

The following option is supported for /usr/xpg4/bin/cp only:

-p

Preserve. cp duplicates not only the contents of source_file, but also preserves the owner and group id, permission modes, modification and access time, ACLs, and extended attributes, if applicable. The command can fail if ACLs are copied to a file system without appropriate support. The command does not fail if unable to preserve extended attributes. If unable to duplicate the modification and access time or the permission modes, cp prints a diagnostic message to stderr and return a non-zero exit status. If unable to preserve owner and group id, cp does not fail, and it clearsS_ISUID and S_ISGID bits in the target. cp prints a diagnostic message to stderr and return a non-zero exit status if unable to clear these bits.

In order to preserve the owner and group id, permission modes, and modification and access times, users must have the appropriate file access permissions. This includes being superuser or the same owner id as the destination file.

When both -p and -@ options are specified, the last specified -p or -@ option determines the behavior.

OPERANDS

The following operands are supported:

source_file

A pathname of a regular file to be copied.

source_dir

A pathname of a directory to be copied.

target_file

A pathname of an existing or non-existing file, used for the output when a single file is copied.

target

A pathname of a directory to contain the copied files.

USAGE

See largefile(5) for the description of the behavior of cp when encountering files greater than or equal to 2 Gbyte ( 2³¹ bytes).

EXAMPLES

---

Example 1 Copying a File

The following example copies a file:

example% cp goodies goodies.old example% ls goodies* goodies goodies.old

---

---

Example 2 Copying a List of Files

The following example copies a list of files to a destination directory:

example% cp ~/src/* /tmp

---

---

Example 3 Copying a Directory

The following example copies a directory, first to a new, and then to an existing destination directory

example% ls ~/bkup /usr/example/fred/bkup not found example% cp -r ~/src ~/bkup example% ls -R ~/bkup x.c y.c z.sh example% cp -r ~/src ~/bkup example% ls -R ~/bkup src x.c y.c z.sh src: x.c y.c z.s

---

ENVIRONMENT VARIABLES

See environ(5) for descriptions of the following environment variables that affect the execution of cp: LANG, LC_ALL, LC_COLLATE, LC_CTYPE, LC_MESSAGES, and NLSPATH.

EXIT STATUS

The following exit values are returned:

0

All files were copied successfully.

>0

An error occurred.

ATTRIBUTES

See attributes(5) for descriptions of the following attributes:

/usr/bin/cp

ATTRIBUTE TYPE	ATTRIBUTE VALUE
Availability	SUNWcsu
CSI	Enabled
Interface Stability	Stable

/usr/xpg4/bin/cp

ATTRIBUTE TYPE	ATTRIBUTE VALUE
Availability	SUNWxcu4
CSI	Enabled
Interface Stability	Standard

SEE ALSO

chmod(1), chown(1), setfacl(1), utime(2), attributes(5), environ(5), fsattr(5), largefile(5), standards(5)

NOTES

The permission modes of the source file are preserved in the copy.

A -- permits the user to mark the end of any command line options explicitly, thus allowing cp to recognize filename arguments that begin with a -.

SunOS 5.10 Last Revised 13 May 2004

O'Relly/Linux Command Directory cp

This directory of Linux commands is from Linux in a Nutshell, 5th Edition.

Click on any of the 687 commands below to get a description and list of available options. All links in the command summaries point to the online version of the book on Safari Bookshelf.

Buy it now, or read it online on Safari Bookshelf.

cp

cp [options] file1 file2 cp [options] files directory

Copy file1 to file2, or copy one or more files to the same names under directory. If the destination is an existing file, the file is overwritten; if the destination is an existing directory, the file is copied into the directory (the directory is not overwritten).

Options

-a, --archive

Preserve attributes of original files where possible. The same as -dpr.

-b, --backup

Back up files that would otherwise be overwritten.

-d, --no-dereference

Do not dereference symbolic links; preserve hard-link relationships between source and copy.

-f, --force

Remove existing files in the destination.

-i, --interactive

Prompt before overwriting destination files. On most systems, this flag is turned off by default except for the root user, who is normally prompted before overwriting files.

-l, --link

Make hard links, not copies, of nondirectories.

-p, --preserve

Preserve all information, including owner, group, permissions, and timestamps.

-P, --parents

Preserve intermediate directories in source. The last argument must be the name of an existing directory. For example, the command:

cp --parents jphekman/book/ch1 newdir

copies the file jphekman/book/ch1 to the file newdir/jphekman/book/ch1, creating intermediate directories as necessary.

-r, -R, --recursive

Copy directories recursively.

-S backup-suffix, --suffix=backup-suffix

Set suffix to be appended to backup files. This may also be set with the SIMPLE_BACKUP_SUFFIX environment variable. The default is ~. You need to explicitly include a period if you want one before the suffix (for example, specify .bak, not bak).

-s, --symbolic-link

Make symbolic links instead of copying. Source filenames must be absolute.

--sparse=[always|auto|never]

Handle files that have "holes" (are defined as a certain size but have less data). always creates a sparse file, auto creates one if the input file is sparse, and never creates a non-sparse file without holes.

-u, --update

Do not copy a file to an existing destination with the same or newer modification time.

-v, --verbose

Before copying, print the name of each file.

-V type, --version-control=type

Set the type of backups made. You may also use the VERSION_CONTROL environment variable. The default is existing. Valid arguments are:

t, numbered

Always make numbered backups.

nil, existing

Make numbered backups of files that already have them; otherwise, make simple backups.

never, simple

Always make simple backups.

-x, --one-file-system

Ignore subdirectories on other filesystems.

Example

Copy the contents of the guest directory recursively into the /archives/guest/ directory, and display a message for each file copied:

cd /archives && cp -av /home/guest guest

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: March 24, 2020