Samba upgrade

One of the medium term projects at work is to move towards Active Directory. Why, you might ask? Supposedly AD would solve some problems that non-Gainesville employees are having, such as password changing. We have a password change after 180(?) days policy and employees on site in many states, so this needs to work.

Anyway, part one is to update our Samba servers to version 3 because AD won't form a domain trust with earlier versions. Last night was the night we updated the Primary Domain Controller. Since I don't know much about Samba, I was mostly along for the ride.

Of course, it's never that simple. Under Samba 2, we were using smbpasswd as the backend, and Samba 3 really wants to have an LDAP backend. We use LDAP for our UNIX accounts, so we just needed to add an ObjectClass for Samba. At least that was the theory. If it wasn't done correctly, the Windows desktops and servers would loose their domain association and everything would come to a stop. This is because Windows domains include a machine account for each machine, which Windows is courtious enough to change the password for periodically. If you don't stop samba on the servers while doing the update, a machine password can be changed while doing the conversion and the mismatch when restarting causes the Windows machine to not be able to communicate with the domain.

This upgrade had been attempted twice before, and despite debugging in a test environment things went poorly and had to be backed out. Due to the previous problems and the need to move forward, it was decided that there was going to be no going back.

The SysAdmin who led the previous attempts at Samba 3 is a full time student at UF and part time with us, so coordinating with him is tricky. Previously he had said everything was essentially ready except for migrating user and machine accounts, which had to be done at the time of the cutover. When he came in around 5pm after classes, he told our team lead what needed to be done before the cutover at 7, then left for some student org meeting and would return after the meeting. It turned out we also needed to write some config files before we could to the upgrade. WTF? #1.

Much much later we had config files and an LDAP update ready. The new configs and LDAP were loaded, and... Success! Mostly anyway. Users were able to log in, Windows machines kept their domain trusts, we just couldn't add new machines to the domain.

We were using Samba 3.0.10, which was the version which shipped with the version of RedHat we were using on the PDC. Prior to 3.0.11, you have to use an account which maps to UNIX UID 0 to add a Windows system to the domain. No problem, a UID 0 account is created, password set, and... No join. Not only no join, but the account had it's sambaLMPassword and sambaNTPassword deleted and the "Disabled" flag added to it's sambaAcctFlags. WTF? says we. Passwords are put back, the account unlocked, and one join later... same thing.

Much muttering, experimenting, and discussing later, the decision was made to try the newest version of Samba 3. It was downloaded and installed. We restart and... nothing works. Cursing ensues. The new Samba is backed out, LDAP restored (from a slave LDAP server) to a version which had been working with Samba 3.0.10, and everything restarted. Things seem to work, other than the machine add script. Things are declared good enough, and we head home at midnight.

Part 2 later...