Understanding WinXP Defragmenting

Not sure if what you want to post fits in the other forums? Post it here!
Mike S.

Understanding WinXP Defragmenting

Postby Mike S. » Fri Aug 10, 2007 11:34 am

Periodically, there are posts about defragmenting hard drives to allow Proshow Gold and Proshow Producer to work better or properly. So I'm posting this to try to help folks understand how WinXP defragmenting actually works and a better solution to defragmenting.

WHAT IS FRAGMENTING?
Start with a new WinXP system on a newly formatted hard drive (HD). Most of the files on the HD are grouped together and the unused space are in vast, large chunks. The following crude diagram illustrates this (where *=files, -=unused space):
******-----------------------------------------------------------------------------------------------------

As the user and system writes files to the HD, the disk looks like this:
******aaaaaBBBBcccDDDDeeeeeeFFFFggggggHHH--------------------------------------------------

Then some files are deleted so the drive looks like this:
******aaaaa-----cccDDDD-------FFFFF--------HHH----------------------------------------------------

Then the user writes a big video file:
******aaaaavvvvcccDDDDvvvvvFFFFFvvvvvvHHH---------------------------------------------------

Note that the "big" video file is written where there were small unused areas so it gets split up into chunks or "fragmented". WinXP knows where every chunk of the video file is so it jumps from chunk to chunk when reading the video file later. This jumping from chunk to chunk is what can slow down the read operation since the HD heads have to physically move to different areas of the HD.


WINDOWS XP DEFRAGMENTING:
Here's what might be a revalation to some. When WinXP does a defragment, it does not move files to one area of the HD (consolidate files). It only defragments the files that are fragmented. So in our example, the files end up looking like this:
******aaaaa------cccDDDD-------FFFFF--------HHHvvvvvvvvvvvvvvvvvvvvvv---------------------

Note that the "small" unused areas remain! The video file is just moved to an area that is big enough to hold it without fragmenting it. If there are no big contiguous unused areas to hold large files, then WinXP moves some files around. This is how WinXP's defragment works. So doing a defragment using WinXP will only allow whatever files currently on the HD to be defragmented. Doing a read operation immediately does indeed make reading the video file faster. But as soon as another big video file is written, it fills the small unused areas and is immediately fragmented.


THIRD PARTY DEFRAGMENTERS:
Years ago when I researched this, I found that one third party defragmenter actually defragments and MOVES (consolidates) files to one area of the HD which leaves all of the unused area in one big chunk. That defragmenter is called "Perfect Disk". After defragmenting using Perfect Disk, the HD looks like this:
******aaaaacccDDDDFFFFFHHHvvvvvvvvvvvvvvvvvvvvvv-------------------------------------------

Now there is a huge contiguous unused space for your video files to be written to without being immediately fragmented.

When I did my research (years ago), it looked to me that "Disk Keeper" did not consolidate files like "Perfect Disk" does. I might be wrong about Disk Keeper but that is how it looked to me from all that I read back then. Even if that was the case then, I don't know if Disk Keeper consolidates files during a defragment now.

SUMMARY:
So if you really want to make defragmenting your WinXP HD worthwhile, I suggest buying a copy of Perfect Disk to do it. There might even be free software now that can do it so if anyone knows of any freebies, please post a link.

Hope this helps understanding WinXP's defragmenting,
Mike S.

.
User avatar
Posts: 7501
Joined: Wed Nov 29, 2006 6:35 pm
Location: Kirkland,Wash, USA, Earth

Postby gpsmikey » Fri Aug 10, 2007 1:18 pm

Yep - Raxco Perfect disk is what I use on my system at home. I have noticed in the past
(although I think it was win2k) that the first time you defragged the drive, you got rid
of the fragments, but still had spaces, but running it several additional times, seemed
to consolidate the files and make for larger free space (the white on the display).
Disk fragmentation, in some circles, seems to be some sort of "religious experience"
and people can get pretty cranked up - some insist it is the best thing since sliced
bread while others will contend it does nothing, is not needed and anybody that does
it is a fool. From my viewpoint, I notice things run faster and smoother after I have
run "Perfect Disk" on my system if it showed things were scrambled around (and as it
defragments, you can see it move the files around on the disk map that is displayed).
Typically, small files do not show the problems with fragmentation since they are
easy to load, but especially large video files can really be helped by defragmentation
since they are so large (and easily fragmented) and the data rate is fairly high when
you are playing them.

Just my $0.02

mikey
You can't have too many gadgets or too much disk space !!
mikey (PSP6, Photoshop CS6, Vegas Pro 14, Acid 7, BluffTitler, Nikon D300s, D810)
Lots of PIC and Arduino microprocessor stuff too !!

ProShow Hall of Fame
User avatar
Posts: 1262
Joined: Sun Oct 01, 2006 7:53 am
Location: New Mexico

Great Informaton Mike

Postby hardsoftware » Fri Aug 10, 2007 1:28 pm

Mikey, there are some folks that won't defragment because they are afraid that they will wear out the drives. :lol:

Mike S, is correct in that DISKEEPER will not consolidate files it deems unmovable. It will not always consolidate free space all together either.

Here are some things I found in the documentation that may or may not address this.


Diskeeper uses a built-in list of files that it will not move under any circumstances. This list is referred to as the Diskeeper internal exclusion list. The most common reason for a file existing on the Diskeeper internal exclusion list is safety. The file cannot or should not be moved in order to maintain the integrity of your system. For example, the memory.dmp file is on the Diskeeper internal exclusion list. This file is on the exclusion list because the disk locations where this file is stored are loaded by the kernel when the system boots and stored in memory. These disk locations are then used to write the dump file should it need to be written. If Diskeeper were to move this file, then the system failed, the dump would be written over other files or in space considered to be free space. This could result in a very corrupted disk.

The same situation applies to hiberfil.sys. When your system goes into hibernation mode, the pre-fetched disk locations are overwritten with the hibernation data. If Diskeeper moved the hiberfil.sys file out and moved other files into those disk locations, then disk corruption would occur. The files boot.ini, NTDETECT.COM, NTLDR, and NTOSKRNL.EXE are not moved or defragmented to ensure that your system remains bootable. (Moving these files to higher locations on a disk can render a system unbootable.) There is additional information in the Microsoft Knowledge Base about why the location of these files is critical.

It is a common misconception that a defragmented disk should look very neat and tidy in the Volume Map tab, with solid blue bars all the way across the screen (representing fragmentation-free files) and the rest white space (representing consolidated space).

Clearly, the speed of the volume (meaning how fast you can access the data on it) is more important than the prettiness of the display or the consolidation of all the free space into one place. Free space consolidation might be important if the next file that you plan to create needs to be one gigantic contiguous file, but it has no effect on performance. In fact, the operating system may or may not write the next file into a contiguous location—even if there is a large enough space.

Because of this, when using the "Quick" or "Recommended" defragmentation methods, Diskeeper uses algorithms that achieve the highest speed from your volumes regardless of the arrangement of the free spaces on the disk and on the screen—and it does so without wasting time on excessive consolidation of free space. We simply go for the fastest possible file access times and then stop.

Even so, you might ask why we don't continue and rearrange the files further to get a neat display? Because it takes computer power to do so. We long ago decided that it would be wrong for Diskeeper to consume more of your computer's performance than it gives back. So Diskeeper defragments until the disk is in top shape performance-wise and then stops.



OK, I've posted this for the information contained within and for no other reason. Really, I don't want to get beat up :lol:
Thanks to Mike for posting his information too because there is a lot of excellent information AND he's done the research.

If you already have Diskeeper you should use it. If not then I would give Mikes suggestion a try. Either one are much better than nothing at all.

Ben 8)
.
PSG, & Producer 3,4 and 5. Photo Editing: PS Elements 2.0 & 5.0, Premier Elements 3.0,

PHOTODEX GALLERY: http://www.photodex.com/share/hardsoftware
UnEmployed, and getting pretty good at it!
Facebook look for Ben R. Baca.

Mike S.

Postby Mike S. » Fri Aug 10, 2007 2:21 pm

Mikey and Ben, thanks for your comments.

My impression of Disk Keeper is that it is just an optimized version of WinXP's defragmenter. It defrags the same way as WinXP but maybe faster.

As Mikey knows since he uses it, Perfect Disk has two basic defragmenting schemes. One is to defragment and consolidate only data files and the other is to include some system files in the defragment. When an analyze is done, Perfect Disk will give you its recommendation of which one it recommends you use. But the user decides, not the software. I'm not sure if Perfect Disk has the option to defragment faster using the WinXP method since I only use the consolidate methods.

FWIW though, I have two dedicated video hard drives and I don't find that it's necessary to defrag either or them to get PSG or PSP to work. I usually do a defrag of my system and video drives using Perfect Disk more as maintenance than to get software to work but that is truthfully few and far between since I'm lazy... :wink:

Mike S.

Return to Odds & Ends

Who is online

Users browsing this forum: No registered users and 19 guests