Tuesday, October 18, 2005
The joy of windows :/
So I took today off from work as an extra day to spend some time with my wife and recouperate from my trip to the startup schoool. Of course, I'm typing this message from work. So much for that extra day off.
We do E-discovery here at work, and we use a collection of tools that imo are worst of breed. We use windows as a file server to house 100's of millions of image and text files. If you think windows doesn't scale well to deal with those numbers, well, you're absolutely right. It doesn't, we have all kinds of problems dealing with the size and number of files. Then we use SQL Server as our database backend. Certainly that's a step forward from Access (which they used to use), but it has all kinds of scale problems for what we are doing, not to mention the horribly designed structures we are torturing it with. Now add to that a visual basic program that does some automation to load other programs and extract data, and you have barely a hint of the problems I deal with at work.
The upside is that I spent most of my time working with Python, and I have an really bright team of people that works with me. Our team uses Python to process text and images, and we are not responsible for the platform decisions we have, we just have to work within them.
So one of our systems runs a C# application designed by another team, and the system is just falling apart. SQL Server 32 bit apparently cannot use more than 1.7 GB of ram without a hotfix. The hotfix says not to do it on a server with only 4 GB of ram. Great. So we bought a Quad Xeon processor with 32 GB of ram from Dell. After some problems we finally get the 64 bit version of windows installed. Whoo..time to get the 64 bit version of SQL Server installed. HA! That only runs on Itanium servers, these are some other whacked 64 bit architecture. wtf?!
Ok, time to install the 32 bit version of SQL Server, so far a complete waste of about 6 hours. After we get it installed, we installed service pack 4 for sql server and the Awe hotfix to allow sql server to address more than 2 gigs of ram. Reboot the server (because that's always a good choice with windows). Ok, to test we set the SQL Server to use a fixed amount of ram rather than dynamically allocate the ram. To test we set it at 10 GB of ram. Now, one might think that the sql server process would then start ramping up it's memory usage and/or spawn a few processes to take up 10 GB right? wrong! It sat there at 128 MB of ram. We tried several different options, none of which worked. At this point my boss suggested we look at Perfmon because he heard you can only see how much memory it's using there. Guess what, none of the SQL Server performance counters are there. Recheck the install, hmm, everything ok there. Reboot the server (because that's a good idea under windows), crap they still don't show up. Apparently the 32 bit counters don't work under a 64 bit OS. Unfortunately the perfmon counters are critical to our performance evaluation, without them we have no idea what sql server is doing.
So now we are down to reinstalling a 32 bit version of windows on the 64 bit quad processor server we have. Of course the night is still young.
Hard to imagine why I refuse to use windows and sql server outside of work eh? If it were my company, Open Source products are the only way to go. Or at the very least, if you want to play in the enterprise, you need to get real enterprise level products. As for me and my house, we'll use Rails on Linux.
Comments: Post a Comment