Sunday, May 03, 2009

Moving Users folder in Vista


Pas de version française, désolé !

(Apologies to regulars, this is not an interesting post!)

Introduction

Now that I've finally managed to do it, my blogging self can't resist the temptation to explain how (in the vein of the unreasonably popular headlamp post).

If you search for this subject, pretty much all paths lead or refer to the same web page on Joshua Mouch's blog, where there is a fairly comprehensive step-by-step how to. However, reading through the (pages of) comments, I noted that there were some unhandled issues with 'symbolic links'. Which is where this post comes in.

Disclaimer

(Read this stuff, it's not your usual yada-yada.)
- This seems to have worked for me, but I've only been running it for couple of days.
- I'm running Vista Home Premium, in French.
- I did this on a new machine, soI had zero data or configuration to lose.
- If you have no way of restoring your system if things go pear-shaped, or you don't trust it enough to test it, venture no further.
- If anything goes wrong, you're on your own, I'm a programmer, not an IT specialist (I had to start over at least 5 times)
- The 'solution' I found involved complications which may not exist on other systems, I've included the extra detail here, which you will have to wade through.
- This post is loooong, sorry. And no pictures (except the totally unrelated introductory picture).
- Broadly-speaking there are three ways of doing this, I only explain one here.
- UPDATE: Upon reflection, I realise why doing this may NOT be a good idea. The plan was to be able to backup and restore the system and data separately. The problem is that the Users folder contains both data AND system stuff. And now that I've got the Users folder on another partition, I can't back it up with the system, which means that if I ever have the disk go down, I'm going to have trouble restoring the users configuration. Ho-hum, MirrorFolder is probably going to be the solution...

Nitty-gritty

You could just skip this bit and go to the instructions, but one is generally well advised to understand what is going on. I do provide a script to automate a part of the grunt work, but it doesn't 'just work' without help.

The basic problem/question is how do I get the Users folder on another drive/partition in Windows Vista? (If you've got to here and you're wondering why anyone would want to do that, you must be really bored!) For me, the main motivation is being able to backup system and data separately. My old PC got progressively slower and slower with time, even though I didn't install that much new software on it. If I can restore the system without losing my data, that means I can always roll back all the bloat/temp stuff, and (hopefully) keep the system nippy. This will mean maintaining a log of all new programs installed, then periodically rolling back to the last backup, and reinstalling the programs and re-doing a backup. Whether I manage to be that disciplined is another question.

Why can't I just cut and paste the Users folder? Well, one problem is that Windows is using bits of it, so you can't move them. The other problem is that even if you did move them, Windows would still be looking for them in the old location, and nasty things happen.

One solution is to modify the installation parameters to set a non-default location for the Users folder. As far as I can tell this is the only clean 'Microsoft' solution. Unfortunately I couldn't do this, because I bought an Acer which just dumps a disk image of a pre-installed Vista.

The solution on Josh Mouch's blog is to 'trick' Windows and progressively move the Users folder a bit at a time. Basically by first telling Windows to put new users in the new location, then logging in with a new user and copying across the other users. As you can see, it's a bit 'dirty', and on top of that, without tweaking it breaks Windows Update.

The other solution that is hinted at in comments is to use the Vista installation disk to 'boot into Recovery mode' and then move everything in one go. This basically means that you get access to all the files and folders, without Windows 'holding' on to them, so you can do what you want. The problem with this method is that 'symbolic links' don't get copied automatically, and you have to recreate them (and their access rights) by hand. I tried this, but at the end couldn't get access to some folders and figured that I'd got it wrong, so re-initialised my system.

So what's a symbolic link? It's kind of a software teleporter. It tells the file system 'from here, go somewhere completely different, but make it look like it's here'. So C:\Documents and Settings doesn't actually exist on disk, it's a symbolic link which points to somewhere else. This is heavily used in Vista to reproduce the 'expected' system paths for older systems, so that old programs will still work. So not reproducing them won't break Vista, but could potentially mean some programs won't work properly.

I must just say in passing that they are all hidden symbolic links, which seem evil to me. I started over 3 or 4 times because I kept ending up with folders that I didn't have access to. It took me a long time to work out that even in normal Vista I didn't have access to the folders, they are part of the backwards-compatibility spaghetti that I shouldn't even have been seeing - except that I had followed the instructions on the blog which told me to change the settings to show all hidden and system files...

So basically, I came up with a script which writes the script to do the copying and recreation of those symbolic links. My script is complicated by the fact that my hard drives are SATA, and the Vista recovery disk didn't have the drivers on, which meant I had to load them separately, which in turn meant that all the disk letters were offset. (Did I mention that it took me a long time to get this working?) This means that my script has to allow for different drive letters between 'normal' running and 'script' running.

Oh the last thing is the really clever bit (not of my invention), which is to create 'one link to rule them all' which makes Windows believe that D:\Users is still in C:\Users...

OK, that said, let's roll...

Instructions

You will need:
- Internet access
- A blank DVD & something to write to it with
- A USB stick
- Potentially, the appropriate disk drivers for your machine
- The script below
- To be comfortable with DOS commands
- To have read these instructions before starting, preferably

1. Download the Vista recovery disc from here (it is a torrent file, so you need to follow the instructions for downloading it, and burning it to disk)

2. Try booting from the DVD, if you see a drive in the list, you are rocking, if not, you're going to have to find the drivers somewhere, copy them onto your USB stick, and hit Load Drivers to go and get them and load them.

3. Select the drive in the list. The next step will invite you to choose a login. If you haven't enabled the Administrator account, I wouldn't use that one.

4. Next, DO NOT hit the big Install button, but at the bottom, click on the Repair option.

5. Now hit the command line link, and have a nosy around to work out what the drive letters of your different drives/partitions are. In my case, I have 5 USB drives, and I had to load the drivers separately, so my C: and D: became I: and J: in recovery mode. This is important!

6. Now boot normally. Open a command prompt and run the following command:
dir /a c:\users > c:\dir.txt
7. This creates a directory listing which the script below will 'eat'

8. Open up the file and check what the prefix of each directory listing is, in English it is " Directory of...", in my case it was " R,pertoire de".

9. Save the script below to a file "script.vbs" at the root of your C: drive

10. Modify the repprefix parameter in the file to reflect the value discovered in step 8 (space included)

11. Check the sourcedrive and targetdrive values. If you had the same driver problem as me in recovery mode, modify the alt values accordingly.

12. Save and run the vbs file (double click it, I think)

13. Have a look at the script.bat file, it starts with a robocopy command, which copies everything except symbolic links (lots of sources on the internet say you can't use robocopy to copy the symbolic links because there is a recursive link under Users. This is true, but even without that link, when robocopy copies links, it creates real folders, not links). Then there are a load of commands which recreate the links one at a time, copy the permissions and set the hidden attribute (confession, I assumed that all the links are hidden, but never checked). Finally, the script renames the Users fold to _Users, and creates a symbolic link from C:\Users to D:\Users.

14. Now boot from the DVD again, do all the same steps (2-5) to get to the command line. Navigate to C:\ (or whatever the equivalent is), and run script.bat.

15. Depending on how much data you have, it could take a while.

16. Note that the script doesn't delete your existing Users folder, it just renames it, so if anything does go wrong, you should be able to delete the symbolic link (rd C:\Users), and rename your Users folder back.

17. One last thing to do, boot back normally, open regedit.exe to edit the registry, navigate to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileList and change %SystemDrive% to D: (or whatever) for Default, ProfilesDirectory and Public. The open ProfileList and correct the ProfileImagePath for the existing users (the ones with long SIDs). This is apparently necessary for Windows Desktop Search.

18. Hope it works for you.

'# Perform dir /a c:\users > c:\dir.txt
'# place this script file in c:\ too
'# double click to run it
'# run resulting script.bat from recovery mode
repprefix = " R" & Chr(130) & "pertoire de " ' Modify to your language
sourcedrive = "C:\"
targetdrive = "D:\"
altsourcedrive = "I:\" 'leave same as target drive unless otherwise indicated
alttargetdrive = "J:\" 'leave same as target drive unless otherwise indicated

inname = "dir.txt"
outname = "script.bat"
juncname = "[jonction]" //After copying from web, replace [] with angle brackets!!!
linkname = "[symlinkd]" //After copying from web, replace [] with angle brackets!!!
userroot = "Users"

set fso = CreateObject("Scripting.FileSystemObject")

' construct batch commands for saving rights, then link, the recreating rights
Function GetCommand(curroot, line, typ, keyword)
' first need to get source and target
pos = Instr(line, keyword) + Len(keyword)

tuple = Trim(Mid(line, pos))
arr = Split(tuple, "[")

oldtarget = Replace(arr(1), "]", "")
oldlink = curroot & "\" & Trim(arr(0))

' need to determine if we are pointing back to old disk
newlink = replace(oldlink, sourcedrive, targetdrive)
if(Instr(oldtarget, sourcedrive & userroot)) then
newtarget = Replace(oldtarget, sourcedrive, targetdrive)
else
newtarget = oldtarget ' still pointing to original target
end if

' comment
out = "echo " & newlink & " --- " & newtarget & vbCrLf
' save permissions
out = out & "icacls """ & replace(oldlink, sourcedrive, altsourcedrive) & """ /L /save " & altsourcedrive & "permissions.txt" & vbCrLf

' create link
newlink = replace(newlink, targetdrive, alttargetdrive)
if typ = "junction" then
out = out & "mklink /j """ & newlink & """ """ & newtarget & """" & vbCrLf
else ' typ = "symlink"
out = out & "mklink /d """ & newlink & """ """ & newtarget & """" & vbCrLf
end if

'set hidden attribute
out = out & "attrib +h """ & newlink & """ /L" & vbCrLf

' apply permissions
shortlink = Left(newlink, InstrRev(newlink, "\") - 1) 'icacls works strangely - non-orthogonal for restore
out = out & "icacls """ & shortlink & """ /L /restore " & altsourcedrive & "permissions.txt" & vbCrLf

GetCommand = out & vbCrLf
End Function

Sub WriteToFile(file, text)
ForWriting = 2
Create = true
set outfile = fso.OpenTextFile(file, ForWriting, Create)
Call outfile.Write(text)
Call outfile.Close()
End Sub

outtext = "ROBOCOPY " & altsourcedrive & userroot & " " & alttargetdrive & userroot & " /E /COPYALL /XJ" & vbCrLf & vbCrLf

set intext = fso.OpenTextFile(inname)
while not intext.AtEndOfStream
line = intext.ReadLine()
if Instr(line, repprefix) then
curroot = Replace(line, repprefix, "")
elseif Instr(line, juncname) then
outtext = outtext & GetCommand(curroot, line, "junction", juncname)
elseif Instr(line, linkname) then
outtext = outtext & GetCommand(curroot, line, "symlink", linkname)
end if
Wend

outtext = outtext & "icacls " & altsourcedrive & userroot & " /L /save " & altsourcedrive & "permissions.txt" & vbCrLf
outtext = outtext & "ren " & altsourcedrive & userroot & " _" & userroot & vbCrLf
outtext = outtext & "mklink /j " & altsourcedrive & userroot & " " & targetdrive & userroot & vbCrLf
outtext = outtext & "icacls " & altsourcedrive & " /L /restore " & altsourcedrive & "permissions.txt"

Call intext.Close()

Call WriteToFile(outname, outtext)

MsgBox("Done writing to " & outname)

13 Comments:

Anonymous Anonymous said...

AH!, So Vista allows you to peep at symbolic links, does it? I knew that they were there somewhere in XP, but could never find exactly how to get at them. Assumed you couldn't. I have always created a stand alone My Documents, My Pictures, My Music etc.., in my preferred location (at root level) then placed a directory short cut to each from the 'Real' User directory. Hit and miss which programs can find them there. Does streamline the backup process, though. Dad.

06 May, 2009 22:20  
Blogger Benjol said...

As far as I know, in XP there aren't any there by default, but the functionality does exist in the FileSystem. There's no way to create them though, without downloading an extra utility (Junction, I believe). And Explorer in XP didn't play nicely with them, I believe (if you deleted the link, it deleted the target, not just the link).

More detail than I could possibly provide here.

06 May, 2009 22:47  
Anonymous Anonymous said...

Hello,
I am the guy, Jay who actually invented the method of moving the users folders for which Josh Mouch took credit for.
Josh Mouch plagiarized the copyrighted article.
Anyway this script you invented is very clever and I am updating my article for Windows 7.
i would like your permission to use your script with credits in my new article.

Please contact me at MRGCAV@gmail.com

07 July, 2009 19:41  
Anonymous Steve Werner said...

This looks just like what I was looking for! I was dreading having to recreate all those junctions!

I saved the code as a vbs script, and tried running it. I got an error message that I hope you can help me with. It said:

---------------------------
Windows Script Host
---------------------------
Script: C:\user location.vbs
Line: 25
Char: 2
Error: Subscript out of range: '[number: 1]'
Code: 800A0009
Source: Microsoft VBScript runtime error

---------------------------
OK
---------------------------


Can you help me correct the error?

Thanks

19 July, 2009 04:25  
Blogger Benjol said...

@Steve Werner: Trying a script copied off internet if you don't understand what it does is incredibly risky!

A couple of pointers: look in the txt file output by the dir command. On my system, the targets of the symlinks are shown in square brackets []. Apparently in your case there aren't any - in any case that is a hypothesis that would explain why the script can't parse out the link targets.

Failing that you'll have to paste a useful snippet of that file here, and I'll see what I can do - (my disclaimer notwithstanding).

21 July, 2009 07:50  
Blogger ShadesOfGrey said...

I found another method of moving the "Users" folder that might help refine your refinement of the method you attribute to Joshua Mouch (read note below). I haven't tried either method yet, so I'm not advocating one over the other. Anyway, the method employed in clockworkrat's blog entry uses "mountvol" to identify the GUID for each drive. FYI, the GUIDs for the 'normal' install of Vista will be different from the "Vista Recovery" CD/DVD or partition. However, MKLINK doesn't validate that the GUID used for the target resolves to an existing disk. It was implied in clockworkrat's blog entry, but I just verified this to be so. The advantage of using the GUIDs for links is that they should help prevent problems with drive re-lettering down the road.

The two potential disadvantages to this method I've found in my limiting testing (for this post). First is that doing a directory listing (DIR /A:L) the "Target" resolves to the GUID and not the drive letter. Of course this is not a big deal unless you've created links that are not going to be hidden from users. Second, since CMD doesn't support UNC paths. If, for some reason, you want to manipulate the Target with an internal CMD command, without using the link, you'll have to use mountvol to discover where the GUID is pointing.


Note:

The method used by MRGCAV@gmail.com & Joshua Mouch is basically a re-hash of a technique for moving the "Documents and Settings" directory under Windows XP. An example of 'prior art' using the techniques used by MRGCAV@gmail.com & Joshua Mouch can be found here, http://support.microsoft.com/kb/314843/en-us%20under.

22 July, 2009 00:36  
Blogger Benjol said...

@ShadesOfGrey, thanks for your interest. Also for the link to MSDN. I didn't find the attribution dispute very interesting anyway, but in view of your prior art reference, it kind of evaporates.

By the way, if anyone is still reading, the new site superuser.com is a good place for these kinds of questions.

22 July, 2009 07:25  
Anonymous Steve said...

Here's my dir.txt file,; I changed all the "< >": to "{ }" so that Blogger won't think they're HTML tags.

Volume in drive C is ACER
Volume Serial Number is 08D7-C0CC

Directory of c:\users

07/16/2009 12:29 PM {DIR} .
07/16/2009 12:29 PM {DIR} ..
11/02/2006 09:02 AM {SYMLINKD} All Users [C:\ProgramData]
11/02/2006 09:02 AM {DIR} Default
11/02/2006 09:02 AM {JUNCTION} Default User [C:\Users\Default]
08/21/2008 08:37 AM 174 desktop.ini
11/02/2006 08:50 AM {DIR} Public
07/19/2009 08:54 PM {DIR} Steve
1 File(s) 174 bytes
7 Dir(s) 5,679,947,776 bytes free

24 July, 2009 00:09  
Anonymous Josh Mouch said...

I am just revisiting my article after these some two and a half years and noticed that Jay (MRGCAV@gmail.com) has been claiming that I plagiarized some article of his. I'd just like to clear my name by pointing you to the following URL:
http://channel9.msdn.com/forums/Coffeehouse/188047-Moving-my-user-folder-to-another-hard-drive-in-Vista/.

You will notice a post by me, and then another post by him, telling my how it was exactly what he was looking for, that it worked perfectly for him, and that he had a few changes that might make it work even better.

If anybody should be accused of plagiarism (which I'm not doing, since I consider this information important enough to be open domain), it's obviously him.

I've written him about this, and hope he will retract his accusations.

20 October, 2009 15:08  
Anonymous Josh Mouch said...

correction:
... telling *me* that it was exactly what he was looking for ...

Also, the article of which I speak is:
http://joshmouch.wordpress.com/2007/04/07/change-user-profile-folder-location-in-vista.

20 October, 2009 15:12  
Blogger adayoldbagel said...

Hey, if this is post is still alive at all, I was having the same problem as steve, so I changed line 25 to read :

"oldtarget = Replace(arr(0), "]", "").

The script ran after that, but the batch file was bogus, because it was mklink-ing the wrong things.

I really wish you'd just email me the contents of your batch file (preferably in english, but if not I'll make due), so that I can just change it to fit my system. My email is a.day.0ld.bag3l@gmail.com

Failing that, maybe you could link me to a site that listed all the symlinks that need to be recreated so that I can just do that manually. Or detail them yourself, I don't care. Please, just answer me.

22 February, 2010 09:59  
Anonymous Pascal said...

Hello Ben,

I'll soon recieve a new machine under Win 7 with a SSD and a HDD. I was initially planning to move the whole C:\Users to the mechanical HD but after reading many sources including your blog and your question on superuser, I'm not convinced this is a that good idea (first, Windows doesn't make the process easy and second, there is actually no clean separation of the OS and user data with Windows, as you mention). So I'm now considering moving only Documents, Videos, Pictures, etc with a simple right click. On a single user machine, would you go this direction if you had to redo it? Thanks in advance for your valuable advice.

29 July, 2011 10:44  
Blogger Benjol said...

@Pascal, I'm guessing you checked the box to receive follow-up comments...

Yes, I'd definitely go with doing it the 'approved' Windows way on the next computer/OS. Especially because of the fuzzy area. Do application settings count as data or not?

I think that the truth is, mostly application settings count as work: if you lose them, it's a bummer, but you can do it all again. The same can't be said of your 'real' data.

02 August, 2011 22:49  

Post a Comment

<< Home

Newer Posts Older Posts