Migration to Exchange Server 2007

We have had our eye on Exchange 2007, a major revision to Microsoft’s leading Mail Server system for some time now. Changes like a Webmail to rival the local Outlook 2003/2007 client, improved Spam filtering and a better administration console we have hearned for a long time not to mention features like Unified messaging ala VOIP integration, Resource Management (which in the future we hope can use to replace our current Resource and Room booking applications and thus elimate one more program off our network) and tight Sharepoint integration which we have just implemented to replace our aging Intranet.

But before we could move to Exchange 2007 we would require new hardware. As I pointed out earlier we were eager to get Exchange 2007 so we had planned this well in advance and thus purchased a new server capable of running our 1000+ mailbox Exchange 2007 setup. Exchange 2007 requires a certain amount of memory per mailbox and a 64bit capable processor as Exchange 2007 only comes in a 64bit flavour to get the best out of the application and server hardware underneath it.

Once we had both the hardware and software ready to go we ran the Exchange Best Practices Analyzer tool and made sure there wasn’t anything dire with our current setup that would have to drastically change before we migrated to the new server. With our current setup checking out we installed Windows Server 2003 R2 64bit on the new hardware following Exchange 2007. This is where we encountered our first error. Unfortunately the media in which our distributor provided us with was a dud and crashes mid way through the installer, lucky enough re-running the installer continues where it left off and finishes the install with no problems. Apparently this problem has happened to many others who have used the same distributor and has now been resolved. Lucky us!

Once we had a fresh install running we setup the inital configuration of the Mailbox, Client Access and Hub Transport Roles and got the server ready to recieve mail once we migrated the mailboxes across. With the initalconfigurations done we set out to migrate the mailboxes across. Surprisingly despite a small hiccup this part is surprisingly easy. I took this chance to get familar with the Exchange Powershell. As the name implies its a shell thats very powerful and specifically for Exchange… The Exchange PS is geared towards people with UNIX background or administrators who have a real need for scripting administrative tasks or just prefer a shell to configure their servers instead of a Graphical User Interface (interestingly enough, every command in the Exchange GUI finishes by showing you the shell commands it has ran on your behalf). After a little bit of research I learned how to migrate them from one server to another and began by migrating the IT Support mailboxes first. Once migrated we tested and prodded to see if everything was okay, including Outlook clients which have already been setup. The migration was a complete successful, no issues presented themselves, all mail, tasks and calenders in tact and access from webmail and the local Outlook client ran flawlessly. With the initial run working perfectly I began migrating student mailboxes across 1 year level at a time while carefully monitoring the transfer for any errors. Once we got through the student mailboxes we moved onto the much larger staff mailboxes. The only problem we ran into was that some mailboxes held more mail that the imposed limit and the transfer would not progress. To get around this we temporally upped the limit on the individual mailboxes on the old server and then migrated the mailbox across to the new server error free.

With all the mailboxes migrated across all the records (like MX records) and connectors to point to the new server and send a couple of emails to and from various mailboxes and made sure email was flowing freely. So far so good. We decided to configure the non critical options like Spam filtering and Postmaster mailboxes a fortnight after the initial migration so we can eliminate any issues before we start locking down the mail server.

The first issue we ran into was our external mail all of a sudden stopped. Mail was still working inside our network but we could not receive any external mail. It ended up being the new server flagged our DMZ box as a spam server and temporally blocked any email being received from that particular server. Now because all external traffic comes from our DMZ box we immediately released why this is happening and put the box in the safe list. Once we did this mail freely flowed again.

The next issue we ran into was Spam Filtering, we when initially configured it in the Exchange System Manager (GUI front end) it basically didn’t do anything. We were quite puzzled over this as we triple checked every option, checkbox etc… and there was nothing wrong. At this point we were getting a ridiculous amount of spam in the majority of mailboxes and our users were becoming quite concerned. We did a bit of research and didn’t find much so we decided to try removing the Spam Filtering and reinstalling it using the Exchange Powershell and hey presto it started filtering messages. After some fine tuning we have it filtering out 99 percent of the spam we receive and successful not blocking any legit messages.

The last problem we encountered were public folders and distribution lists. Unfortunately some Outlook client were not receiving some of the DLs in our mail server and others receive a error when sending and receiving that points to distribution of pubic folders. Microsoft’s Knowledge Base informs us that Service Pack 1 for Exchange 2007 shall fix both these issues. We have scheduled a time during the school holidays in 3 weeks to perform this upgrade so we have the time and no interruptions if something does go wrong.

All in all it hasn’t been a stressful migration. Apparently 2000 to 2003 was much more painful but I shall never know

Migration to Windows Vista

The inevitable did happen. Late last year we had a student enter the laptop program with a laptop with Windows Vista on it. Needless to say we were unprepared for it but we could not make the student turn around the ask the retailer to downgrade their laptop to Windows XP as this would be reoccurring theme in the near future not to mention we pride ourselves in being a technically proficient College. Willing to accept the challenge we took this opportunity to see how our current setup of software, configurations and policies worked in a Vista environment.

First of the rank, our set of software on the client performed near flawlessly, nothing in Vista prevented any of our application suite from functioning typically. Aside from a few issues of license issues where application installed and configure on a local account did carry over its settings to any new account created on the laptop (namely our domain accounts); nothing could of not gone more smoothly.

The only major software issue we came across was Ghost 8.0′s reluctance to successfully clone a HDD with Windows Vista installed on it. This was due to Windows Vistas GUID Partition Table system which replaces the Master File Table used in previous version of Windows. A simple (HA!) upgrade to the latest Ghost removed this issue.

Our configurations on the other hand were no so lucky. Machine-based Wireless Certificates did not work the same way they did in Windows XP (which required a Registry hack to work anyway) and forced us to use User-based Wireless Certificates which was quite painful as this meant the wireless connection did not connect until a User had successfully logged into the laptop which in turn meant no logon scripts would run. It was not until the middle of semester 1 until someone in another school figured out the inner workings of Vista’s Wireless and the new Network & Sharing Center and thus enabling Machine-based Wireless Certificates.

To work around this issue we provided teachers of Year 7 classes documentation on navigating to the Network Shares via UNC paths (as there wireless worked, just not quick enough for logon scripts).

Our policies took an even worse beating. Printers deployed via Group Policy seemed sporadic at best, sometimes all the printers showed up, sometimes none with no ability to add them via the Add Printers function in Windows. Eventually Service Pack 1 cleared this problem right up. Luckily this issue only seemed to effect 1 Lab of computer in the Library. What was different about those particular ones in comparison to other labs and our latest laptop roll out is quite bemusing.

Some users network shares were also affected by Vista. Instead of being d:\Student\ExitYear\Usernameon the Fileserver Vista would rename their folder so it looked like d:\Student\ExitYear\Documents. Luckily the network share name was unaffected and users could continue to access their share, so it was more an administration issue particularly when we wanted to go through student folders. This problem is documented well by Microsoft and unfortunately the problem will not go away till there are no Windows XP machines accessing the shares.

So in the end the laptop was quite helpful in finding out little quirks and familiarizing ourselves with Microsoft’s new operating system. But to get ready for mass deployment on our desktops we needed to evaluate our current range of desktops and see if they are capable of running Vista to our standard. In comes Microsoft’s Windows Vista Hardware Assessment Tool, a handy little application I discovered in my Windows Vista Client for Enterprise Support Technicians Training Kit. This application ‘is an inventory, assessment, and reporting tool that will find computers on a network and determine if they are ready to run the Windows Vista™ operating system or the 2007 Microsoft Office System’. In running this tool we discovered the majority of our current PCs would not handle Windows Vista to our standard, most could only just handle the Basic aspects of Vista and we did not want the students experience of Vista to be a painful one so with this information as ammunition we planned with our ICT Coordinator to upgrade our Library Labs hardware to specifications capable of running Vista. The Lab adjacent to the IT Support office would also be upgraded because the Hardware was very new and passed the WVHA Tool comfortably.

We scheduled 3 days for deployment of the new hardware in the Library and successfully deployed the new hardware with Windows Vista and Office 2007.

The next step was to monitor the new lab and gain feedback from Students and Staff. Although we are still continuing to do so the only change we made apart from deploying Service Pack 1 to remove the teething issues (see above) was utilizing the Microsoft provided ADM files in Group Policy to force all the Office packages to save by default in the 2000-2003 File Format to prevent students not being able to access their work that was saved on the new Library lab on computers that do not have Office 2007, namely other labs, their home computers and their teachers laptops.

Generally the method is to Plan>Test/Evaluate>Deploy>Monitor/Gain Feedback but in this case we evaluated the software before we planned to the deployment but I guess you have to be flexible and work with the situation at hand.

Tuesday 11th September

Today we laid the groundwork for a project to allow the Basketball Association (who are located in the shared Basketball Stadium which resides of School property) to use our infrastructure to obtain internet access. Currently they uses 56k Dial Up as ADSL over copper lines is unavailable in this location and satellite is far too expensive for an small/low profit organisation such as the Basketball Association.

For the Basketball Association to use our infrastructure to gain fast internet access provides three obstacles:
1. They will create add to the traffic on our infrastructure (performance)
2. They will be open to our network and vice versa (security)
3. They will need to be accurately charged for the access (cost)

Problems 1 & 2 are easily solved by segmenting the traffic from the rest of the schools traffic using Virtual Local Area Networks (or VLANs). And because we already have VLANs implemented inside the school network it is very very easy to add another VLAN. And because the traffic is seperate from the rest by using the VLANs we can apply security rules just to that particular VLAN to prevent any unwanted traffic or snooping.

Problem 3 is a seperate issue. At first the gateway between the schools infrastructure and their infrastucture was going to be our ex firewall, a Watchguard FIREBOX (see figure 1) and we would use a linux solution to count the traffic and charge accordingly but we happened to stumble upon a CISCO 1700 series router (see figure 2) and used it to be the gateway between the two networks.

I have never had experience setting up a router of this capability so I left that upto my colleague but watched on. They run the same CISCO IOS system that the switches dotted are the school run so it was fairly (or more so exactly) similar command wise but because this is a router it had various features the Layer 2 switches did not (the Layer 3 switch is capable of routing so it too was very similar to the router). One thing my colleague had to implement was a Loopback interface on the Router. He did this so the charging of internet traffic would not charge the Basketball Association for traffic such as DHCP broadcasts and pings. Using ACLS once traffic hit the Ethernet 0 Interface all traffic would be routed to the Loopback Interface 0 where broadcast traffic would hit a dead end so to speak (a null value) and the rest of the traffic would be sent back to the router’s processor and eventually Ethernet 1 and onto the Internet.

Figure 1

Figure 2

Monday 10th September

Today there were no outstanding jobs or any pressing matters so I provided ad-hoc support to the laptop users throughout the day.

Friday 7th September

Today we discovered by a gloating student that Skybe was still managing to work in our network. To combat this with the help of Google I delve into the net to find out the various methods that Skybe uses to connect to the internet while my colleague tailed the students internet traffic. We both discovered that when Skybe cannot connect traditionally using TCP ports (and the GET command) it creates a SSL tunnel (using the CONNECT command) to bypass any proxy authentication methods.

To block this pest (a clever one at that) we create a Access Control List which blocks any method of CONNECT that connects to a IP address (outright blocking connect would make all SSL based websites like secure banking or external webmail cease to work) as Skybe does not use DNS names to connect the Skybe clients to one another. A detailed write up can be found at http://lists.grok.org.uk/pipermail/full-disclosure/2005-November/038646.html and all credit goes to the creator of that document to discovering how to block Skybe without blocking legitimate traffic.

Thursday 6th September

It was pretty quiet today so we decided to be proactive and check the pods and labs around the school and make sure all the desktops were operational. Unfortunately there were a few mice missing from desktops dotted around the school and we were still awaiting a shipment of mice so we could not replace the missing ones. We removed the keyboard from the desktops with missing mice to prevent kids stealing the keyboards (which is a issue in itself).

After checking the pods we tailed the proxy access logs and looked for proxy websites which the kids use to get around the blocks in place by both our department and by our ISPs filtering department. In the past we used ISA 2000 as our internet proxy software which provided decent reporting applications but since moving to Squid we can see what internet sites everyone is going to live and ban the site on the spot. Couple this with Squid-Stats which provides numerous statistics including Top 30 users/sites/exits which gives us an overall view and SARG which provides history down to the user level and we have complete control over internet access in the school.

Wednesday 5th September

Today the ladies in the Learning Centre brought down a print out from their HP Laserjet that resides in their classroom and pointed out it is starting to print out lines on every page. Past experience tells that the majority of time this means the toner is out or faulty so I replaced the toner and got them to print out a lengthy document. No lines were evident on the print out so my job completed.

Tuesday 4th September

Today we decided we would have a crack at preventing kids from playing games against each other over the network. We did some research and discovered whats called MAC ACLs or Media Access Control (address) Access Control Lists. Using the MAC ACLs we can restrict traffic from just the desktops to the VLAN gateway and thus preventing communication between the desktops themselves. Now this can only be applied to the desktop VLAN as other devices such as print servers and Wireless Access points need to communicate with many devices besides the VLAN gateway. Unfortunately we are yet to be able to prevent wireless users from playing games against each other but the biggest offenders come from desktop users so we can consider this successful for the time being.

Because of this type of restriction we decided to implement it on a trial basis and only implement it on the local switches so we can monitor it effects on day to day usage of the PCs (if any). If we are convinced that it will not provide any issues to the desktop users we will implement MAC ACLs across the entire switching infrastructure at Dromana SC.

Monday 3rd September

Today there wasn’t any issues outside the norm so I was providing ad-hoc support for laptop users throughout the majority of the day.

Friday 31st August

This morning the Library staff called up our department requesting some assistance with plugging the VCE only computers back into the network. The night before they had a presentation and had to move the computers to make room for all the people in attendance. They reported all the computers were reporting the domain CURRIC was unavailable and they could not log into the PCs. When I went upto the Library the first thing I check was for the green link light on each of the PCs to make sure it wasn’t a cable issue. I checked all 6 and all 6 had a solid green link light which was unexpected. Next up I logged into each of the PCs using a account that did not have cached credentials and all 6 computers successfully. I asked the Library staff if they plugged in the ethernet cable after they powered on the PCs which their reply was yes. This was the problem as the ethernet adapter takes a while to sync up if plugged in after the PC starts up.