Build 7.0.0.2 (64-bit) odd errors /disk utilization inquiry
-
- Junior Member
- Posts: 11
- Joined: Fri Jan 17, 2014 8:02 pm
Build 7.0.0.2 (64-bit) odd errors /disk utilization inquiry
I've been using EDP for a few years now after it was recommended to me by a dev at work across various system builds and have been extremely happy with the product .
A bit of background so you know where I am coming from:
- I'm using EDP primarily for manual data backups via diff analysis after running into a data-loss nightmare with a syncronization utility (and haven't looked back =)
- I just upgraded to Windows 8.1 Pro 64-bit (clean install) from Windows 7 Ultimate 64-bit and migrated my data onto new hard drives. On Windows 7 I was only using version 6 builds fyi (without a single issue on any stable build)
- My backup procedure is as follows:
> The bulk of my data (845GB atm) is sitting on a RAID 1 hybrid mirror (I'm using an ASUS Sabertooth 990FX mobo which has an onboard Promise hybrid RAID controller) and I do weekly differential (incremental) backups via EDP to a removable hard drive (hot-swap eSATA drive bay). A NAS/automated backup solution is on my wish list
> I have a "junk" drive (193GB of data atm) that the data I manipulate gets written to so as to decrease the wear on my primary OS SSD (Samsung 256 GB 840 Pro) via excessive write cycles and fragmentation on my primary data array (Seagate 3TB ST3000DM001's) that is mostly for archival/persistent data
- After using the native Windows 8.1 file manager to copy my data off my old array + junk drives to the new drives I purchased I ran a file comparison against the source + destinations to make sure everything was copied over 100% completely (yes I'm paranoid, but working in IT as a sysadmin for over 10 years I learned I had to be so I could sleep at night =D )
> While I was running the first comparison I noticed a major system performance hit (it almost felt like a lockup but was just a slowdown) which seemed unusual as I was working with a data set not located on the OS drive. My system has 16GB of DDR3 1600 and an AMD FX-9370 8-core CPU @ 4.4 GHz/core so I thought this was rather odd. On my older system build (running only a 4-core AMD Phenom II 965 @ 3.4 GHz/core and 16GB of DDR3 1333) I could still work while an EDP comparison was running. I pulled up the Task Manager and Resource Monitor and saw my CPU utilization sitting at roughly 50% which seemed fine as well as the 2 HDD's being utilized at 100% sequentially and concurrently.
> The major concern I had was looking at the disk utilization on my OS SSD (Samsung 840 Pro) that was maxxing out at 100% which is ALOT of bandwidth and data writes I am trying to avoid (not sure 100% if these were reads or writes only). This was the cause of the system-wide slowdown obviously in combination with my system ram bouncing around 100% (16GB) utilization. I let the comparison finish which took a very long time compared to what I was used to in the past dealing with the same data set on my older system build.
> Also I received a couple of errors after the comparison completed that I have not seen in the past. These errors were strange as they were referencing files sitting on my OS SSD. This should not have been the case as I used the Windows Explorer "Select First Item to Compare" integrated command on the 2 directories I was comparing.
Any ideas or recommendations? I apparently can't use this:
ExamDiff Pro 6.0
Latest Build: 6.0.3.13 (Saturday, November 02, 2013)
For Windows 98, Me, 2000, XP, 2003, Vista, 7, 2008, 8, 2012.
As Windows 8.1 is not on the compatibility list...
A bit of background so you know where I am coming from:
- I'm using EDP primarily for manual data backups via diff analysis after running into a data-loss nightmare with a syncronization utility (and haven't looked back =)
- I just upgraded to Windows 8.1 Pro 64-bit (clean install) from Windows 7 Ultimate 64-bit and migrated my data onto new hard drives. On Windows 7 I was only using version 6 builds fyi (without a single issue on any stable build)
- My backup procedure is as follows:
> The bulk of my data (845GB atm) is sitting on a RAID 1 hybrid mirror (I'm using an ASUS Sabertooth 990FX mobo which has an onboard Promise hybrid RAID controller) and I do weekly differential (incremental) backups via EDP to a removable hard drive (hot-swap eSATA drive bay). A NAS/automated backup solution is on my wish list
> I have a "junk" drive (193GB of data atm) that the data I manipulate gets written to so as to decrease the wear on my primary OS SSD (Samsung 256 GB 840 Pro) via excessive write cycles and fragmentation on my primary data array (Seagate 3TB ST3000DM001's) that is mostly for archival/persistent data
- After using the native Windows 8.1 file manager to copy my data off my old array + junk drives to the new drives I purchased I ran a file comparison against the source + destinations to make sure everything was copied over 100% completely (yes I'm paranoid, but working in IT as a sysadmin for over 10 years I learned I had to be so I could sleep at night =D )
> While I was running the first comparison I noticed a major system performance hit (it almost felt like a lockup but was just a slowdown) which seemed unusual as I was working with a data set not located on the OS drive. My system has 16GB of DDR3 1600 and an AMD FX-9370 8-core CPU @ 4.4 GHz/core so I thought this was rather odd. On my older system build (running only a 4-core AMD Phenom II 965 @ 3.4 GHz/core and 16GB of DDR3 1333) I could still work while an EDP comparison was running. I pulled up the Task Manager and Resource Monitor and saw my CPU utilization sitting at roughly 50% which seemed fine as well as the 2 HDD's being utilized at 100% sequentially and concurrently.
> The major concern I had was looking at the disk utilization on my OS SSD (Samsung 840 Pro) that was maxxing out at 100% which is ALOT of bandwidth and data writes I am trying to avoid (not sure 100% if these were reads or writes only). This was the cause of the system-wide slowdown obviously in combination with my system ram bouncing around 100% (16GB) utilization. I let the comparison finish which took a very long time compared to what I was used to in the past dealing with the same data set on my older system build.
> Also I received a couple of errors after the comparison completed that I have not seen in the past. These errors were strange as they were referencing files sitting on my OS SSD. This should not have been the case as I used the Windows Explorer "Select First Item to Compare" integrated command on the 2 directories I was comparing.
Any ideas or recommendations? I apparently can't use this:
ExamDiff Pro 6.0
Latest Build: 6.0.3.13 (Saturday, November 02, 2013)
For Windows 98, Me, 2000, XP, 2003, Vista, 7, 2008, 8, 2012.
As Windows 8.1 is not on the compatibility list...
Last edited by EDP_Zomb1e on Sat Jan 18, 2014 1:49 pm, edited 2 times in total.
Re: Build 7.0.0.2 (64-bit) odd errors /disk utilization inq
I'm not sure what your directory comparison options are but the default for 7.0 is full file comparison (Options | Directory Compare) . This involves opening each file in each matching pair, preprocessing it (which sometimes causes a temporary file to be written), and comparing the pair. I suspect you were using a different option in your previous version of ExamDiff Pro.
There's also a new option under Options | Directory Compare | Other, "Use parallel full comparison", which allows multiple threads to perform full comparison faster. This explains using 50% of your CPU, which is by design. You can disable this option and switch to pre-7.0 way of comparing (if you indeed need full comparison); it's slower but more predictable.
There's also a new option under Options | Directory Compare | Other, "Use parallel full comparison", which allows multiple threads to perform full comparison faster. This explains using 50% of your CPU, which is by design. You can disable this option and switch to pre-7.0 way of comparing (if you indeed need full comparison); it's slower but more predictable.
psguru
PrestoSoft
PrestoSoft
-
- Junior Member
- Posts: 11
- Joined: Fri Jan 17, 2014 8:02 pm
Re: Build 7.0.0.2 (64-bit) odd errors /disk utilization inq
My directory comparison options in the version 6 builds were default with no configuration and I'm using the same with v7. I just jump directly to the directory I'm looking to compare and use the "Select First Item to Compare" queing method or launch EDP and run a new comparison and browse the directories. It sounds like the directory comparison options default was different in v6 EDP builds as I don't recall ever changing this.
Here is my current config (default):
http://i1284.photobucket.com/albums/a57 ... 0ff3ac.jpg
That makes sense now about seeing the disk utilization on the EDP installation hard drive (C).
http://i1284.photobucket.com/albums/a57 ... 8bd7d9.jpg
I tried running EDP as Administrator (although my account on Win 7 and 8.1 was already Admin) but this made no difference with the errors. I thought it might be a directory permissions issue.
I received a number of popups similar to below in both scenarios (I never saw this on Win 7/ version 6 builds):
I checked the directory above and indeed there is a ton (over 600 instances of word, excel. tmp and other) of files with the "ed" prefix so I just manually purged these, however why via the parallel full comparison method was EDP not able to purge the temp files created on its own?
Also, I let one comparison run for a few hours against two folders with around 75GB of data and the compare never seemed to complete, although my system's cpu/disk/memory utilization was next to nil when I checked back in on it.
Here is my current config (default):
http://i1284.photobucket.com/albums/a57 ... 0ff3ac.jpg
That makes sense now about seeing the disk utilization on the EDP installation hard drive (C).
http://i1284.photobucket.com/albums/a57 ... 8bd7d9.jpg
I tried running EDP as Administrator (although my account on Win 7 and 8.1 was already Admin) but this made no difference with the errors. I thought it might be a directory permissions issue.
I received a number of popups similar to below in both scenarios (I never saw this on Win 7/ version 6 builds):
I checked the directory above and indeed there is a ton (over 600 instances of word, excel. tmp and other) of files with the "ed" prefix so I just manually purged these, however why via the parallel full comparison method was EDP not able to purge the temp files created on its own?
Also, I let one comparison run for a few hours against two folders with around 75GB of data and the compare never seemed to complete, although my system's cpu/disk/memory utilization was next to nil when I checked back in on it.
Last edited by EDP_Zomb1e on Sat Jan 18, 2014 1:47 pm, edited 2 times in total.
Re: Build 7.0.0.2 (64-bit) odd errors /disk utilization inq
Default 6.0 options would be like this:
Which translates into the following in 7.0:
Please make sure this is what you have. We'll be looking into full comparison errors (after all, 7.0 is in still in Beta).
Which translates into the following in 7.0:
Please make sure this is what you have. We'll be looking into full comparison errors (after all, 7.0 is in still in Beta).
psguru
PrestoSoft
PrestoSoft
-
- Junior Member
- Posts: 11
- Joined: Fri Jan 17, 2014 8:02 pm
-
- Junior Member
- Posts: 11
- Joined: Fri Jan 17, 2014 8:02 pm
Re: Build 7.0.0.2 (64-bit) odd errors /disk utilization inq
It took 10 seconds to run the directory comparison this time =)
Re: Build 7.0.0.2 (64-bit) odd errors /disk utilization inq
Perfectly understandable: EDP is not going inside files in this mode.It took 10 seconds to run the directory comparison this time =)
BTW, is there a way for you to zip your folders and attach here, so that we could debug the errors you received?
psguru
PrestoSoft
PrestoSoft
-
- Junior Member
- Posts: 11
- Joined: Fri Jan 17, 2014 8:02 pm
Re: Build 7.0.0.2 (64-bit) odd errors /disk utilization inq
This is perfect! Exactly the configuration I need for my requirements. For my backup purposes I don't need to modify files, only copy/delete/replace, although in other scenarios I do but a file vs directory comparison works fine for that. Thanks for the fast response considering it's the weekend
Glad to support PrestoSoft - you guys are top notch!
Let me know if you need any help testing the Beta moving forwards.
Cheers!
Glad to support PrestoSoft - you guys are top notch!
Let me know if you need any help testing the Beta moving forwards.
Cheers!
Re: Build 7.0.0.2 (64-bit) odd errors /disk utilization inq
Thanks. You probably didn't see the edit of my last reply but here's what I asked: is there a way for you to zip your folders and attach here, so that we could debug the errors you received? Attaching your options exported to a file will also be helpful.
psguru
PrestoSoft
PrestoSoft
-
- Junior Member
- Posts: 11
- Joined: Fri Jan 17, 2014 8:02 pm
Re: Build 7.0.0.2 (64-bit) odd errors /disk utilization inq
Sure, I will figure this out
-
- Junior Member
- Posts: 11
- Joined: Fri Jan 17, 2014 8:02 pm
Re: Build 7.0.0.2 (64-bit) odd errors /disk utilization inq
Sorry for the extended delay - was going through some massive system unstability (AMD driver/hardware issues...)
After the latest update I no longer received the errors with the default configuration options (I was checking then moved to your recommended settings which work fine). However in respect to housekeeping, I revisited the directory (C:\Users\?\AppData\Local\Temp) where I found the original logs and temp files piling up. It seems the latest version of ExamDiff Pro does not clean up after itself still. Do you require the temp files? I opened them in Notepad++ and they are data directly from the dataset I checked.
After the latest update I no longer received the errors with the default configuration options (I was checking then moved to your recommended settings which work fine). However in respect to housekeeping, I revisited the directory (C:\Users\?\AppData\Local\Temp) where I found the original logs and temp files piling up. It seems the latest version of ExamDiff Pro does not clean up after itself still. Do you require the temp files? I opened them in Notepad++ and they are data directly from the dataset I checked.
Last edited by EDP_Zomb1e on Thu Apr 03, 2014 3:53 pm, edited 2 times in total.
Re: Build 7.0.0.2 (64-bit) odd errors /disk utilization inq
Could you confirm this? Please delete all ed*.tmp files from the temp folder, run a comparison, close ExamDiff Pro and check the existence of ed*.tmp files. Make sure you have no running ExamDiff Pro instances at that point (you can use the Task Manager to verify).
psguru
PrestoSoft
PrestoSoft
-
- Junior Member
- Posts: 11
- Joined: Fri Jan 17, 2014 8:02 pm
Re: Build 7.0.0.2 (64-bit) odd errors /disk utilization inq
Comparison is underway. It's a 21GB dataset so it will take a few hours comparing recursively. BRB =)
-
- Junior Member
- Posts: 11
- Joined: Fri Jan 17, 2014 8:02 pm
Re: Build 7.0.0.2 (64-bit) odd errors /disk utilization inq
First, is it the latest build (7.0.0.9)? Second, I was under the impression you where using metadata comparison, based on the the thread history.
Temp files are not deleted if there was a failure during comparison that was not correctly handled (and you do get errors). It's hard to tell the exact root cause without having your files, which, given the size of the data, is not possible.
If file contents comparison is not needed in your case, use metadata comparison, it's much faster.
Temp files are not deleted if there was a failure during comparison that was not correctly handled (and you do get errors). It's hard to tell the exact root cause without having your files, which, given the size of the data, is not possible.
If file contents comparison is not needed in your case, use metadata comparison, it's much faster.
psguru
PrestoSoft
PrestoSoft