FTP mirroring

[Siliconjunkie]

I currently have a FTP server running on Tru64 with about 2000 users. Each has their own directory that is owned by them.

I will be migrating them to a new Linux box in the coming weeks and need to divise a way to move all the data AND retain permissions and ownership.

Here is where it gets tricky. The boxes are in 2 different datacenters and they basically have no way to talk to each other.

The only reasonable option is to use a go-between host to pull the data off of the old one and push it up to the new one.

So, what I am thinking is a tool that will do FTP mirroring that will remember owner/permissions.

If that is not possible I will have to come up with a script to chown and chmod the directories. This shouldnt be hard since the directory name is the same as the user/group name.

Oh, and tarring MAY be an option. Part of the reason for the move is disk space. I might be able to move some of the big clients before hand.

Any ideas?

[Mntsnow]

Whew...I'm going to have to do some research on this before I suggest anything...

[Siliconjunkie]

Here is my current idea:

Move the data all at once, owner/permissions be damned.

Then, on windows, using a batch file create a script to chown and chmod all the dirs.

[Siliconjunkie]

ok, for some reason posting cleared my mind....and it came to me

I have a list of all the boxes called "boxes.lst"

If I do this:

for /f %i in (boxes.lst) DO echo chown %i /home/%i -R >> changeown.sh
I get:

chown box1 /home/box1 -R
chown box1 /home/box2 -R
chown box1 /home/box3 -R

Change this up a bit to change owner, group and permissions and I end up with 3 simple scripts that will fix all the permissions, owners and groups. Or 1 long one to do all 3. I think that trying to do a chown owner:group will break the for command. May try it tho. Can't hurt.

This is the great part about being familiar with multiple OSes you can use each where you are strong in them. Who would have thought using a windows batch file to create a shell script.

[Siliconjunkie]

Well I just noticed a serious flaw.

chown box1 /home/box1 -R
chown box1 /home/box2 -R
chown box1 /home/box3 -R

keeps changing it to box1.... needs some tweaking, but not at 3 am on my day off

[Mntsnow]

Yeah posting at 3am and then on your day off...Your Sick...You best go back to bed! ;)

[Siliconjunkie]

Nah, thats the joy of being off weekends. Just got home from bowling.

[Mntsnow]

How did you make out on this SJ? Would like to know what you ended up doing so I can have it in my "bag of tricks" if I ever need to do that myslef :)

[Siliconjunkie]

Well, we found a utility to import the passwd file from the Tru64 box, so we didnt have to recreate the accounts. The scripts I whipped up took care of ownership and permissions.

On the moving of data, it became to 2 part process with a catch.

This is a going forward plan right now, the schedule for this actually happening goes thru 1/1/2004.

1) Let the users have access to the new box and legacy box. Let them take care of moving the data. Luckly the data is not static, so we only assure it will stay there for 3 days. This helps with 90% of the users.

2) To move the ones that have massive amounts of data 5-6 gig and need to move it easily (only about 5 users) here is what I devised.
a) From 3rd Linux box do this:
wget ftp://root:pass@old.server.com/path/to/user/data -r
This does a recursive wget on the users folder and basically creates a local mirror.

b) to get the data to the new server:
scp /path/to/local/data root@new.server:/path/to/new/data -r
to copy it up there via scp.
It breaks premissions/ownership. But for the small # that need it, thats no problem.

So, effectively what we have done is to put the burden of moving data on the users. Now, the hurdle of turning them loose on the new one, and decomissioning the DNS name of the old one. This will break all of the automated processes that use it. So, we still have a few hurdles to overcome.
Building a web interface for them to change their password and a few other small things. All this darn security with shadow passwords and such make that a real bear.

[Mntsnow]

Thanks for the info SJ :)