Links and attributes broken when editing on remote shares

[Gingko]

Hello,

I would like to submit a problem that may concern a limited number of users, but can make ExamDiff Pro almost unusable for those who want to use it on the following way:

I very commonly edit, from my Windows XP machine, files that are actually located on a Linux remote machine, using network sharings.

These files have usually some attributes like owners, file permissions, soft and hard links, etc.

When I save modified files using ExamDiff Pro, it appears that a new file is created, eventually leaving the old one renamed as a ".bak" file.

I know this is a very common pratice for making a secure saving of the file, but I think it is wrong for that reason:

That way, the new file completely looses its file permissions, owners, hard and soft links (which are now set on the .bak file), and it is created with the default permissions and owners (and no links) defined for new files created through remote shares.

For avoiding this problem, the files have probably to be saved on the very same file handle that was used for reading them, and it is the backup file that would have to be created as a new file.
(by the way, having an option to create these backup files in different directories and/or volumes would be also quite pratical)

I know several editing tools that had previously the same problem, and have finally it fixed, probably on user requests.

Gingko

(P.S: I tested this with ExamDiff Pro 3.4 evaluation)

[MudGuard]

Have you tried to turn off the creation of backups?

View - Options - File Editing, remove check from "Create backups (*.bak) when saving files"

[Gingko]

Have you tried to turn off the creation of backups?

View - Options - File Editing, remove check from "Create backups (*.bak) when saving files"

Yes.

The result is the same.
(backup files are probably just deleted after saving using this option)

Gingko

[psguru]

The result is the same. (backup files are probably just deleted after saving using this option)

True. The actual culprit is not backup files but the fact that when a file is saved, it is first written to a temp file, then the temp file replaced the original file. Now, we do restore security attributes of the original file but overall trying to reproduce all of metadata Windows stored for the original is probably a losing battle. Here's an idea: instead, implement a feature similar to the one in Textpad:

Overwrite original files directly, when saving changes.
For safety, TextPad can save changes to a temporary file, before deleting the original and renaming the temporary file. This loses the original file creator and attributes. By checking this option, the original file is overwritten, which preserves the creator, at the risk of losing the original file if the save operation fails. This setting is ignored for FAT file systems, which have no concept of a file creator.[/code]
What do you guys think?

by the way, having an option to create these backup files in different directories and/or volumes would be also quite pratical

Again, borrowing form Textpad, perhaps the following options will be useful:

Backup FILE.EXT as:
-FILE.BAK
-FILE.BAK.EXT
-FILE.EXT.BAK
-FILE.EXT in folder: <input for folder>

[Gingko]

The result is the same. (backup files are probably just deleted after saving using this option)

True. The actual culprit is not backup files but the fact that when a file is saved, it is first written to a temp file, then the temp file replaced the original file.

That is more or less what I was saying except that I had more or less the implicit idea that the backup file and the temp file may be actually the same file, and when there is a backup file, it can be just the temp file renamed as a backup file at the end of the saving process (this looks like to me a natural way to do the things ...)

Now, we do restore security attributes of the original file but overall trying to reproduce all of metadata Windows stored for the original is probably a losing battle. Here's an idea: instead, implement a feature similar to the one in Textpad:

Overwrite original files directly, when saving changes.
For safety, TextPad can save changes to a temporary file, before deleting the original and renaming the temporary file. This loses the original file creator and attributes. By checking this option, the original file is overwritten, which preserves the creator, at the risk of losing the original file if the save operation fails. This setting is ignored for FAT file systems, which have no concept of a file creator.

What do you guys think?

It is true that overwriting the original file directly is less safe than creating a new file, that's why I thought of the idea of creating the temp/backup file first, as an exact copy of the original file, then overwriting the original file (using or not the temp/backup file depending if all the file can be kept in memory or not), and finally either deleting the temp file or renaming it as a true backup file depending if the user wants a backup file or not.

by the way, having an option to create these backup files in different directories and/or volumes would be also quite pratical

Again, borrowing form Textpad, perhaps the following options will be useful:

Backup FILE.EXT as:
-FILE.BAK
-FILE.BAK.EXT
-FILE.EXT.BAK
-FILE.EXT in folder: <input for folder>

I didn't know Textpad. I was using another powerful text editor named EditPlus. I just made a Google search for Textpad, and I briefly looked at their site. After a very short comparison, it looks like that Textpad and Editplus could be very similar, and for the purpose of the features that you quoted, one could have simply just replaced "Textpad" by "Editplus" for the same result.

I never have had any problem with creators, links and permissions on remote shares with EditPlus.

Gingko

[MudGuard]

That is more or less what I was saying except that I had more or less the implicit idea that the backup file and the temp file may be actually the same file, and when there is a backup file, it can be just the temp file renamed as a backup file at the end of the saving process (this looks like to me a natural way to do the things ...)

This would require three operations with the complete file:
1. read the original file,
2. write the original file as backup,
3. write the changed file.

Renaming the original file and writing the changed file under the name of the original file only requires one operation with the complete file.

[Gingko]

This would require three operations with the complete file:
1. read the original file,
2. write the original file as backup,
3. write the changed file.

Renaming the original file and writing the changed file under the name of the original file only requires one operation with the complete file.

Of course, but it is not usable on remote shares ...

Gingko

[psguru]

I thought of the idea of creating the temp/backup file first, as an exact copy of the original file, then overwriting the original file (using or not the temp/backup file depending if all the file can be kept in memory or not), and finally either deleting the temp file or renaming it as a true backup file depending if the user wants a backup file or not.

In this scenario, however, if save fails, the original file restored from the temp file will not get back its security attributes (see Remarks in http://msdn.microsoft.com/library/defau ... pyfile.asp).
The same is true if you use the unsafe saving option ("Overwrite original files directly, when saving changes") in Textpad: if the save fails, the original file will be left corrupted.

Also, once again, EDP does restore the original file's attributes (using GetFileAttributes/SetFileAttributes) and security attributes (using GetNamedSecurityInfo/SetNamedSecurityInfo). So perhaps another way to attack this issue is to take care of remaining attributes, and for that I need to know they are. You mentioned creators, links, and permissions on remote shares but I need to know exactly what works and what doesn't, for both local drives and remote shares.

I never have had any problem with creators, links and permissions on remote shares with EditPlus.

I don't know how EditPlus does saving. Perhaps they use the "unsafe" saving, and you never had a failure.

Backup FILE.EXT as:
-FILE.BAK
-FILE.BAK.EXT
-FILE.EXT.BAK
-FILE.EXT in folder: <input for folder>

This was my attempt to address your request ("by the way, having an option to create these backup files in different directories and/or volumes would be also quite pratical"). I'm still not sure if this feature is needed -- I haven't received any feedback from you on this.

[Gingko]

In this scenario, however, if save fails, the original file restored from the temp file will not get back its security attributes [...]

Maybe, but I think that it is certainly less complicated to have to manually reset attributes after a failure than to have to recreate the whole file from scratch
(but it is really very very boring to have to manually reset attributes after every file saving ....).

The same is true if you use the unsafe saving option ("Overwrite original files directly, when saving changes") in Textpad: if the save fails, the original file will be left corrupted.

Also, once again, EDP does restore the original file's attributes (using GetFileAttributes/SetFileAttributes) and security attributes (using GetNamedSecurityInfo/SetNamedSecurityInfo). So perhaps another way to attack this issue is to take care of remaining attributes, and for that I need to know they are. You mentioned creators, links, and permissions on remote shares but I need to know exactly what works and what doesn't, for both local drives and remote shares.

Actually, my remotes shares are not very often NTFS or FAT volumes. I use to access "ext2" or "ext3" (Linux) partitions from a Windows XP machine via Samba in order to updates files on a Linux server. I am not sure if GetNamedSecurityInfo/SetNamedSecurityInfo can work properly in that configuration ...

I don't know how EditPlus does saving. Perhaps they use the "unsafe" saving, and you never had a failure.

Editplus is not the only software that properly preserves attributes in that case. Compare It! and WinMerge handle them the same way (although it is a very recent fix in Compare It!).

If you think that this can be really unsafe, you can just give to users the choice of using one method or the other using a configuration option ....

This was my attempt to address your request ("by the way, having an option to create these backup files in different directories and/or volumes would be also quite pratical"). I'm still not sure if this feature is needed -- I haven't received any feedback from you on this.

I wouldn't like to look like making proselytism about Editplus, but EditPlus does it that way also ...

Gingko

[psguru]

OK, I'll need some time to think this through. Thanks for your input...

[Gingko]

OK, I'll need some time to think this through. Thanks for your input...

Ok ...

I would just like to add to this that one of my particular concerns in this matter are links. And especially hard links.

For those who don't know what are hard links (they are not very well known in the Windows world), they are files that have multiple directory entries on the same volume of the same file system, but they all occupy the same space on the disk.

It is a little like making a shortcut, but it is much more sticky: in fact, there is no difference between the original file and its hard links; you can move both the file and the hard link and they are still linked together.

If you delete the original file, the hard link will be still there and valid, and actually you will be just like if hou had just moved the original file to the hard link location.

If you have multiple hard links and you very want to delete the file, you must delete the file and all of its hard links : the file will be actually deleted only after the last of its links will have been deleted.

Hard links are mostly part of the Unix/Linux (ext2/ext3) filesystems, and I use to create them there, but they are also possible on NTFS (not FAT) filesystems; try the following in command line (cmd.exe) mode :

Code: Select all

fsutil hardlink create <hard_link_name> <original_file_name>

You can also read this and this about them.

Hard linked files appears to you exactly like if it was the exact copy (including attributes and permissions) of the same file on all places where there are hard links.

But you will see that if you modify this file, the changes will be seen on all places.

... And that is the problem : if, instead of directly modifying the file, you create a new file for writing the changes, and after that you delete the old file, you will completely break the hard link. The new file will become orphan in regard of its previous link(s), and other "instances" of the link will be left unchanged as only one of its link will have been deleted.

This is usually not what it is expected: I mostly create hard links not only because that allow me to use less space on disks, but also because I want to be able to update the contents of all of them at the same time. In my case, that allows me to have the same file on multiple web sites, and to be able to change all of them at once.

Gingko

[psguru]

The latest build of 3.4 Beta contains new options: to save a file directly, and to provide more ways to save backups. See them under Options | File Editing.

[zweistein]

I'm not sure what options were there before, but given the opportunity, I chose "FILE.EXT in the following directory" and typed the same relative directory as in Dir Comparison > More > Subdirectory for backup. Which may not be what was intended, but there's no complaint about the directory not existing. But when saving an edited file C:\folder\file.ext, this message pops up:

Could not save file C:\folder\file.ext. The system cannot find the file specified. (Error 2)

It's not clear at all where EDP is trying to write what here.

[psguru]

I see. That directory was supposed to be an absolute path but you raise a good point. Will add support for relative paths in the next build of 3.4 Beta.