PrestoSoft Community

I've been using EDP for a few years now after it was recommended to me by a dev at work across various system builds and have been extremely happy with the product .

A bit of background so you know where I am coming from:
- I'm using EDP primarily for manual data backups via diff analysis after running into a data-loss nightmare with a syncronization utility (and haven't looked back =)
- I just upgraded to Windows 8.1 Pro 64-bit (clean install) from Windows 7 Ultimate 64-bit and migrated my data onto new hard drives. On Windows 7 I was only using version 6 builds fyi (without a single issue on any stable build)



- My backup procedure is as follows:
> The bulk of my data (845GB atm) is sitting on a RAID 1 hybrid mirror (I'm using an ASUS Sabertooth 990FX mobo which has an onboard Promise hybrid RAID controller) and I do weekly differential (incremental) backups via EDP to a removable hard drive (hot-swap eSATA drive bay). A NAS/automated backup solution is on my wish list
> I have a "junk" drive (193GB of data atm) that the data I manipulate gets written to so as to decrease the wear on my primary OS SSD (Samsung 256 GB 840 Pro) via excessive write cycles and fragmentation on my primary data array (Seagate 3TB ST3000DM001's) that is mostly for archival/persistent data
- After using the native Windows 8.1 file manager to copy my data off my old array + junk drives to the new drives I purchased I ran a file comparison against the source + destinations to make sure everything was copied over 100% completely (yes I'm paranoid, but working in IT as a sysadmin for over 10 years I learned I had to be so I could sleep at night =D )

> While I was running the first comparison I noticed a major system performance hit (it almost felt like a lockup but was just a slowdown) which seemed unusual as I was working with a data set not located on the OS drive. My system has 16GB of DDR3 1600 and an AMD FX-9370 8-core CPU @ 4.4 GHz/core so I thought this was rather odd. On my older system build (running only a 4-core AMD Phenom II 965 @ 3.4 GHz/core and 16GB of DDR3 1333) I could still work while an EDP comparison was running. I pulled up the Task Manager and Resource Monitor and saw my CPU utilization sitting at roughly 50% which seemed fine as well as the 2 HDD's being utilized at 100% sequentially and concurrently.
> The major concern I had was looking at the disk utilization on my OS SSD (Samsung 840 Pro) that was maxxing out at 100% which is ALOT of bandwidth and data writes I am trying to avoid (not sure 100% if these were reads or writes only). This was the cause of the system-wide slowdown obviously in combination with my system ram bouncing around 100% (16GB) utilization. I let the comparison finish which took a very long time compared to what I was used to in the past dealing with the same data set on my older system build.
> Also I received a couple of errors after the comparison completed that I have not seen in the past. These errors were strange as they were referencing files sitting on my OS SSD. This should not have been the case as I used the Windows Explorer "Select First Item to Compare" integrated command on the 2 directories I was comparing.

Any ideas or recommendations? I apparently can't use this:

ExamDiff Pro 6.0
Latest Build: 6.0.3.13 (Saturday, November 02, 2013)
For Windows 98, Me, 2000, XP, 2003, Vista, 7, 2008, 8, 2012.

As Windows 8.1 is not on the compatibility list...

I'm not sure what your directory comparison options are but the default for 7.0 is full file comparison (Options | Directory Compare) . This involves opening each file in each matching pair, preprocessing it (which sometimes causes a temporary file to be written), and comparing the pair. I suspect you were using a different option in your previous version of ExamDiff Pro.

There's also a new option under Options | Directory Compare | Other, "Use parallel full comparison", which allows multiple threads to perform full comparison faster. This explains using 50% of your CPU, which is by design. You can disable this option and switch to pre-7.0 way of comparing (if you indeed need full comparison); it's slower but more predictable.

My directory comparison options in the version 6 builds were default with no configuration and I'm using the same with v7. I just jump directly to the directory I'm looking to compare and use the "Select First Item to Compare" queing method or launch EDP and run a new comparison and browse the directories. It sounds like the directory comparison options default was different in v6 EDP builds as I don't recall ever changing this.

Here is my current config (default):
http://i1284.photobucket.com/albums/a57 ... 0ff3ac.jpg

That makes sense now about seeing the disk utilization on the EDP installation hard drive (C).

http://i1284.photobucket.com/albums/a57 ... 8bd7d9.jpg

I tried running EDP as Administrator (although my account on Win 7 and 8.1 was already Admin) but this made no difference with the errors. I thought it might be a directory permissions issue.

I received a number of popups similar to below in both scenarios (I never saw this on Win 7/ version 6 builds):




I checked the directory above and indeed there is a ton (over 600 instances of word, excel. tmp and other) of files with the "ed" prefix so I just manually purged these, however why via the parallel full comparison method was EDP not able to purge the temp files created on its own?

Also, I let one comparison run for a few hours against two folders with around 75GB of data and the compare never seemed to complete, although my system's cpu/disk/memory utilization was next to nil when I checked back in on it.

Default 6.0 options would be like this:
Which translates into the following in 7.0:
Please make sure this is what you have. We'll be looking into full comparison errors (after all, 7.0 is in still in Beta).

Here's the default options that I had under "Directory Compare":



Using your example above and will recompare the last directory set I received the errors with.

It took 10 seconds to run the directory comparison this time =)

It took 10 seconds to run the directory comparison this time =)

Perfectly understandable: EDP is not going inside files in this mode.

BTW, is there a way for you to zip your folders and attach here, so that we could debug the errors you received?

This is perfect! Exactly the configuration I need for my requirements. For my backup purposes I don't need to modify files, only copy/delete/replace, although in other scenarios I do but a file vs directory comparison works fine for that. Thanks for the fast response considering it's the weekend

Glad to support PrestoSoft - you guys are top notch!

Let me know if you need any help testing the Beta moving forwards.

Cheers!

Thanks. You probably didn't see the edit of my last reply but here's what I asked: is there a way for you to zip your folders and attach here, so that we could debug the errors you received? Attaching your options exported to a file will also be helpful.

Sure, I will figure this out

Sorry for the extended delay - was going through some massive system unstability (AMD driver/hardware issues...)

After the latest update I no longer received the errors with the default configuration options (I was checking then moved to your recommended settings which work fine). However in respect to housekeeping, I revisited the directory (C:\Users\?\AppData\Local\Temp) where I found the original logs and temp files piling up. It seems the latest version of ExamDiff Pro does not clean up after itself still. Do you require the temp files? I opened them in Notepad++ and they are data directly from the dataset I checked.

Could you confirm this? Please delete all ed*.tmp files from the temp folder, run a comparison, close ExamDiff Pro and check the existence of ed*.tmp files. Make sure you have no running ExamDiff Pro instances at that point (you can use the Task Manager to verify).

Comparison is underway. It's a 21GB dataset so it will take a few hours comparing recursively. BRB =)

Still getting those errors actually and the temp files are not auto-purged. Here are some screenies documenting this. Let me know if you want the temp files I will zip them up and throw them in my dropbox.

First, is it the latest build (7.0.0.9)? Second, I was under the impression you where using metadata comparison, based on the the thread history.

Temp files are not deleted if there was a failure during comparison that was not correctly handled (and you do get errors). It's hard to tell the exact root cause without having your files, which, given the size of the data, is not possible.

If file contents comparison is not needed in your case, use metadata comparison, it's much faster.