Missing update

[MSpagni]

Very often I copy bunches of files from a panel to the other ("Copy to other directory").
Sometimes (relatively often) it happens that the copy is done perfectly but the status of some of the files is not correctly updated in the panes.
The files are still marked as different or present in only one directory.

I can't suggest a bulletproof method to reproduce the problem.

N.B. This problem is not specific of the new beta.
Didn't I already told you about this problem?
I think so but I can't remember exactly.

Edit: Yes indeed. Here: viewtopic.php?f=3&t=573

[psguru]

I can't reproduce this. In the original thread you had very detailed steps:

* compare a pair of trees with the "sort both directories" option, e.g. by path, so as to have "holes" in the left panel due to "added" files.
* mark a block of consecutive files with mixed status in the left panel list, including some holes, and copy it to the other directory.
* the copy is done perfectly but the status of the last n files in the left panel, with n equal to the number of "holes", is wrong (e.g. they are still shown as "changed").

Since there's no "both" options anymore, I used the "Combine first and second directory sort results" option, which is an extension of the former. If you have specific steps, please post them here.

[MSpagni]

If this time I didn't specify the steps is because I simply can't.
I'm not able to find any pattern. Sometimes it happens, sometime it doesn't.

Sometimes copying a single file, sometimes copying multiple files (but only few of them show the problem)...
Sometimes copying files that overlap older versions, sometimes copying files that doesn't exist in the othe directory...

As soon as I discover more I'll keep you informed.

N.B. In the meantime I'm migrating to a newer computer and a "new" operating system, that is, from my win 2000 on a pentium III 800 to XP on a core duo 3160. I'll tell you what happens.

[MSpagni]

I still haven't any clue about that.

What I can say now is that both my computers present the same problem.

I suppose it can be dependent on the options I use.
I didn't change them lately so they are very similar to the ones I already sent you, but if you prefer to refresh them I can send you the latest version.

I don't think it's important, but usually I copy the files using the right mouse button menu.

[psguru]

Yes, please resend your options. How many file (or directories) do you normally copy when this happens?

[psguru]

Increasing the "Time difference to ignore" setting under Dir Comparison | Other. Try making it 2 (the default value) or 3 seconds should fix this problem. This number is used, in addition to its main purpose, to determine whether a file was actually coped. For more on this see http://www.prestosoft.com/edp_faq.asp#8

[MSpagni]

We are betting on the wrong horse, I fear.
Please read.

EDP Time difference to ignore: 0
Comparing directories (a directory branch).

File A.txt on FAT32: 36.262 KB 02/17/2010 19:46:36 Changed
File A.txt on NTFS: 36.256 KB 02/17/2010 21:59:25 Changed

Copying file A.txt via EDP from NTFS to FAT32.
Copy ok, but EDP says they are still different.

EDP says:
NTFS: A.txt 36.256 KB 02/17/2010 21:59:25 Changed
FAT32: A.txt 36.262 KB 02/17/2010 19:46:36 Changed
(same as before!)

An external file manager says:
NTFS: A.txt 36.256 KB 02/17/2010 21:59:26
FAT32: A.txt 36.256 KB 02/17/2010 21:59:26

Another EDP instance comparing the two files says "equal".
Comparing the two files inside the same instance of EDP (but another window, of course) says "equal".

Copy again.
This dialog appears:
"Do you want to replace the existing file
35,4 KB modified today 02/17/2010, 19:46:38
with this file?
35,4 KB modified today 02/17/2010, 19:46:36"

Answer: Yes.
Copy ok, but EDP says they are still different (same as before!)

Copy again.
Same dialog as before.

Answer: Yes.
Copy ok, but EDP says they are different(same as before!)

etc. etc.

After recompare: directories are identical.

Then: who cares about the timestamps?
The timestamps can be different, but I have checked "Consider files with same size and timestamp identical", NOT "Consider files with different timestamps different" nor "Full file comparison" (I'm very positive about that).

The options for evaluating two files a "equal" can be different between the directory comparison and the file comparison, but this can not explain the effect.

It is apparent that it's EDP that doesn't update its data: look at the file size!
The timestamp has nothing to do with this problem.

[psguru]

I don't think EDP is inconsistent. Considering your options and the following:

This dialog appears:

"Do you want to replace the existing file
35,4 KB modified today 02/17/2010, 19:46:38
with this file?
35,4 KB modified today 02/17/2010, 19:46:36"

here's the explanation: when you copied your file, the destination timestamp became 2 seconds earlier that the source. When EDP verifies that a file was indeed copied (the Windows Shell API, unfortunately, does not provide such functionality), it checks a couple of things: it makes sure that the sizes are identical, and that either the destination time is greater or equal to the source time, or, if it's less, it's less by no more than a certain number of seconds (taken from the "Time difference to ignore" option). Since your option is set to 0, EDP thinks that the file was not copied successfully. Recomparing makes the new file show up with the correct time, and its status is set based on your comparison options. Hence the default for this option is 2 seconds, and I see no reason, given the fact that you compare NTFS and FAT32 volumes, to change this.

[MSpagni]

it checks a couple of things: it makes sure that the sizes are identical, and that either the destination time is greater or equal to the source time, or, if it's less, it's less by no more than a certain number of seconds (taken from the "Time difference to ignore" option).

Ok, this is a satisfacctory explication.
Since that is the heuristic, this explains why the problem appears and how to solve it.

Thank you and sorry for the PITA.

[AlexL]

I did not try to reproduce the problem, but the following fragment is a bit suspicious:

...
File A.txt on FAT32: 36.262 KB 02/17/2010 19:46:36 Changed
File A.txt on NTFS: 36.256 KB 02/17/2010 21:59:25 Changed

Copying file A.txt via EDP from NTFS to FAT32.
Copy ok, but EDP says they are still different.

EDP says:
NTFS: A.txt 36.256 KB 02/17/2010 21:59:25 Changed
FAT32: A.txt 36.262 KB 02/17/2010 19:46:36 Changed
...

If I understand correctly, this means that after COPY operation has finished, EDP did not get the proper (updated) status of the files on that specific FAT32 volume.

Most of other lines of the previous post (that I omitted) are not so relevant (and maybe are even distracting), because they show behaviour of other tools, which were activated several seconds after COPY has finished, and these tools simply have read the updated status from their start. They don't need timestamp update, because they were launched after the COPY.

But the quoted fragment may point to some problem.

[MSpagni]

But the quoted fragment may point to some problem.

That's what I was thinking too but, since it appears that there is no "official" way to detect if the copy went ok, you have to resort to heuristics, as is explained in a previous message.
Now, we can argue that there are cases like that in which the heuristic fails and perhaps can be made better, but I think the actual condition is acceptable, once understood the what, how and where.

Do you have a better idea?

[AlexL]

But the quoted fragment may point to some problem.

...since it appears that there is no "official" way to detect if the copy went ok...

This is also questionable. You don't need to know if copy finished ok. You only need to know when to update panels (it may happen several times even before COPY finished - along the process when COPY writes parts of the file and its attributes; or it may happen when COPY finished with failure; etc.). And I did not encounter problems with asynchronous panel update in some other applications (such as FAR - such file manager which I use every day). I did not try to program such applications, but I always thought that Windows API provides reliable way to get information about file system update.

However, such application (in our case it's EDP) may fail to update its panels properly if it makes some optimizations during the process. For example, if it tries to avoid multiple panel updates during the process of COPY operation, and tries to update only at the end, and only the updated parts, but has a bug in the algorithm of ignoring intermediate updates.

[psguru]

This is also questionable. You don't need to know if copy finished ok. You only need to know when to update panels (it may happen several times even before COPY finished - along the process when COPY writes parts of the file and its attributes; or it may happen when COPY finished with failure; etc.). And I did not encounter problems with asynchronous panel update in some other applications (such as FAR - such file manager which I use every day). I did not try to program such applications, but I always thought that Windows API provides reliable way to get information about file system update.

Unfortunately, it's not the case for the Shell API, which EDP uses in order to support file recycling.

However, such application (in our case it's EDP) may fail to update its panels properly if it makes some optimizations during the process. For example, if it tries to avoid multiple panel updates during the process of COPY operation, and tries to update only at the end, and only the updated parts, but has a bug in the algorithm of ignoring intermediate updates.

As you correctly noted, the goal of this heuristic is to avoid status updates of items that were not copied. This is especially relevant to cases when a large number of file copies was started and then cancelled. I suppose we could, instead of the "Time difference to ignore" option, use a different setting, or simply hardcode it to 2 seconds (which appears to be a safe number) but this a rarely an issue.

[AlexL]

...I suppose we could, instead of the "Time difference to ignore" option, use a different setting, or simply hardcode it to 2 seconds (which appears to be a safe number) but this a rarely an issue.

I'd like to return your attention to the following fragment again:

...
File A.txt on FAT32: 36.262 KB 02/17/2010 19:46:36 Changed
File A.txt on NTFS: 36.256 KB 02/17/2010 21:59:25 Changed

Copying file A.txt via EDP from NTFS to FAT32.
Copy ok, but EDP says they are still different.

EDP says:
NTFS: A.txt 36.256 KB 02/17/2010 21:59:25 Changed
FAT32: A.txt 36.262 KB 02/17/2010 19:46:36 Changed
...

In such case no setting would help, because - according to the original poster - the fact of copying was not noticed by EDP at all.

[psguru]

Perhaps I'm misunderstanding your point. All this says is that, after the file was copied, EDP did not update the view. And the reason has been explained in this thread.