Tuesday, August 24, 2010

Shrinking the Information Store in Exchange When ESEUTIL Doesn't Do It

I recently ran into a problem where after running ESEUTIL /d, the Information Store was still over the 75GB limit that you can expand to in Exchange 2003 Standard SP2. As a refresher, the total size of the Information Store is the EDB + STM, this is what Exchange looks at when it calculates the size of the Information Store.

I'll list a general outline that most people do to clear up "whitespace" in Exchange.

  1. Increase the Information Store limit in Exchange via the registry: http://www.petri.co.il/change_store_size_limits_ex2003_sp2.htm
  2. Archive and then Delete unneeded mailboxes in Exchange
  3. Run an offline Defrag: Unmount Information Store and run eseutil /d infostore.edb
At this point, the information store is usually reduced and users and admins are happy. But what if the information store is still too big, or too close to the limit? What if you just want to know the size of "whitespace" on the information store before you do the defrag?


Check to see how much whitespace you have in the information store.

Whitespace is the empty space left over in the information store after data has been removed from it. I like to compare it to someone getting really fat, and then all of a sudden losing a lot of weight. Once this happens, there's a lot of loose skin. The loose skin needs to be tightened up again. Well the "space" between the loose skin is "whitespace", and we need to remove it to make the body tight again. Yes, weird analogy, but it's how I remember it. Let's get to how to check whitespace in Exchange 2003. A good article on whitespace can be found here: http://www.msexchange.org/articles/Exchange-Databases-Disk-Consumption.html

1. Navigate to where you have eseutil installed (if you haven't adjusted the path so you can type it anywhere in DOS)
2. Go into Exchange System Manager ("ESM"), and dismount the target 4store.
3. Run the following command: eseutil /ms infostore.edb > sizecheck.txt
/m = filedump mode
/s = streaming file
> = send the output to
4. Check out the file sizecheck.txt, and look a line that says "Free", note the number, and the last number at the end of the file, but before the line that says "Operation Completed in 'x' seconds", note this number too.
Free: 24093
234234

The first number is the SLV Space Dump for the STM file.
The second number is the Space Dump for the EDB file.

Take the SLV Space Dump File and multiply by 4096 bytes
Take the Space Dump for EDB and multiply by 4096 bytes

ex.
24093 x 4096 bytes = 98684928 bytes = 94.1132812 megabytes of Whitespace
234234 x 4096 bytes = 959422464 bytes = 914.976562 megabytes of Whitespace

914.976562 mb
+ 94.1132812 mb
---------------------------
1009.0898432 mb = 0.9854393 gigabytes ~ 1GB

As you can see, by running this offline defrag check, we would be able to regain about 1GB of space off of our Information Store if we ran a real offline defrag. It's important to note that the commands above do not defrag the Exchange Information Store, it's read-only and only calculating and dumping statistics. But, what happens if we still need more space? One clue to this is by adding up the EDB and STM files and comparing that to the sum of the size of all the mailboxes. If you're lucky, you'll see that the sum of all the mailbox sizes is LESS than the information store size. Hmmm....what's taking up the rest of the space then? The answer is Recoverable Items.

Recoverable Items

Recoverable Items are really deleted-deleted items. You have an email, you delete it, and it goes to the deleted items folder. You empty you're deleted items. You have just deleted-deleted items. But wait...They aren't deleted. Oh, Snap!

In Exchange 2003 Recoverable Items, aka "Deleted Items Recovery" is enabled by default. It's 7 days for deleted email, and 30 days for deleted accounts. If you do the numbers, this can be a whole lot of email that's added up. So, how do you add it all up? You don't have to, Microsoft will do it for you, in the MMC.

1. Go to Performance Monitor
2. Remove all the items there by clicking on "-"
3. Click on the "+" to add an item
4. Find MS Exchange Mail, then choose Total Size of Recoverable Items
5. Click on the paper with the lines on it to give you numbers. This will tell you the size of recoverable items in the information store in kilobytes.

Another old school way, is to open up ESM. Click on Mailboxes. Go to view, and add the "Deleted" tab. You can now see exactly what user tends to have a lot of deleted items, or add all the mailboxes up manually to get the total sum of recoverable items taking up space on the Information Store.


Removing Recoverable Deleted-Deleted Items

Now you know how to find out how much whitespace you have on the Information Store and how much Recoverable Items space is in the Information Store. How do you regain the space on the Recoverable Items? Well, just doing an offline defrag will NOT clear this up. To get this done really soon across the entire information store:
1. Open up ESM
2. Right click on the Mail Store, and go to Properties
3. Find the Limits tab, and click on it
4. Change "Keep deleted items for" and "Keep deleted mailboxes for" to "0" for both. Click Apply.
5. Find the Database tab.
6. In Maintenance Interval area, select "Custom" and change it to the soonest time you'd like.

Summary / Explanation

What will happen now is that the online maintenance will completely wipe all Recoverable Items off the face of the Earth. The place where the Recoverable Items were in the Information Store, now turns into whitespace. You'll know it's done once the online maintenance reports that it's done in Event Viewer. Once done, run an offline defrag to regain space. Hopefully your Information Store is shrank considerably. Don't forget to readjust the Recoverable Items policy back to something helpful (a balance between recovering data, and recovering space). If this happens again, or you see the Information Store growing again, you've outgrown Exchange 2003 Standard and are ready for Exchange 2003 Enterprise or greater. For kicks, while you're running the offline defrag to regain the whitespace from the Recoverable Items, you can refresh ESM while viewing the mailboxes with a viewing column of deleted, and you'll see them shrink before your eyes as you refresh your view.

BTW, if for whatever reason it's not reasonable to do this across all mailboxes, you can adjust the retention policy for individual mailboxes by going in Active Directory Users and Computers ("ADUC") and adjusting the retention policy on a per user/mailbox basis, and then run the online maintenance, rinse, wash, repeat.

Sunday, March 14, 2010

Use Your Acer Aspire One as an eBook Reader

This will probably be a series of posts explaining what you can do to make your Aspire One an ebook reader. This is for the version of the Aspire One that uses Linpus for those of us out there still running it.

Some things to get out of the way:
I want to be upfront about this. You cannot install any of the programs that work for the Nook or Kindle. There are some workarounds though, but to date they still don't work for the DRM that gets setup with ebooks sold for Nook proper or Kindle proper.

In the meantime, here are some tips of what you can do to read ebooks on your Acer Aspire One.

1. Install Wine

To do this, install Wine by running: yum install wine
So far so good. I haven't had any problems with the older version it installs, and it hasn't ruined any of the applications.

2. Install eReader on Wine For pdb Files

eReader is an application that allows you to read the eBooks that are DRM'd from eReader.com. Wine is a an application that allows you to run some Microsoft Windows programs in Linux. eReader.com is a Barnes and Noble book store. The Nook application for Windows is based on eReader, but doesn't run quite well on Wine. As an option, you can buy ebooks from eReader.com and read them on eReader under Wine (the DRM requires you to input your CC# and Name on your Credit Card once to open the eBook).

Download eReader for Windows
1. Open up Terminal and unzip eReader: unzip "eReader Win Pro 3.0.3.zip"
2. Open up Terminal and run the setup for eReader: wine setup.exe
3. Follow the Windows prompts to install it in Linux
4. Once done you can open it up from /home/user/.wine/drive_c/Program Files/Palm Digital Media/eReader. You can either "cd" into there, or run the string above with quotes like this: wine "/home/user/.wine/drive_c/Program Files/Palm Digital Media/eReader" and it will open. If you download any pdb files copy them to "drive_c" and any directory therein and point eReader to that place, and it will use that directory as your library catalog.

3. Use Adobe Acrobat for PDFs

If you have a PDF of an eBook that's not DRM'd, you can use the built in Adobe Acrobat app to read the pdf'd eBook.

4. Use ePub Catalog in Firefox for reading epub files

Open Firefox and install the EPUBREADER Extension.
https://addons.mozilla.org/en-US/firefox/addon/45281
You can now read ePub files in Firefox and catalog them in Firefox.

5. Rotate Your Screen

Yes, you CAN rotate your screen in Linux for reading eBooks so that your screen is more like a Kindle or Nook.

In Adobe Acrobat:
1. Open your PDF
2. In Acrobat go to: View > Rotate Screen > Counter Clock Wise
3. In Acrobat go to: View > Full Screen Mode

To revert back, click on the escape key on your keyboard ("Esc") and then rotate the screen clock wise.

You can now "turn pages" by clicking the down arrow (it will be pointing left if you hold the netbook with the mouse pad in your left hand). Optionally, the right arrow, which is the down arrow if your holding your netbook as an ebook reader will work too.

In Everything Else:
1. Open Termnal
2. Type in: xrandr -o left

Wine
1. Open Terminal
2. Type in: winecfg
3. Select "Graphics". Then select "Emulate a Virtual Desktop"
4. Change resolution to 600 x 1080
5. Select Apply, then OK.

Open the apps as normal after you've rotated your screen. It can get a little confusing navigating the screen once you've rotated it because you've rotated EVERYTHING on the screen, but the mouse navigation is still the same (if you go up on the mouse, it goes left if you've changed your screen orientation to face left).

To return back to normal: xrandr -o normal

Sunday, February 21, 2010

Twitter Stats

Interesting, I didn't know I was one of the first people to twitter in Southern California (#15). http://twitterholic.com/top100/joined/bylocation/Southern%20Cali/


I also didn't know I am ranked #41 in the update category for Southern California. http://twitterholic.com/top100/updates/bylocation/Southern%20Cali/


Overall stats: http://twitterholic.com/dexterous1/ I'm ranked #2,544,961 :P

Cell Signal Extender




A while back I bought a cell signal extender. It turned out that the one I bought didn't work too well in my area. I had bought a zBoost YX500PCS. This works in the upper frequencies . It escapes me what frequency it was at, but it worked well for 3G internet connections, but not for phone calls (which I cross referenced on forums and it turns out that cell carriers sometimes split data and voice to take up more power on a certain frequency). I'm on Verizon Wireless, and in my area it seems VZW works on the ~800MHz range.


At any rate, I sold it on eBay and got a YX500CEL. I tested using it and saw a bar or two more on my Droid. When i placed a call, it would jump to four bars. My data went from RTTx1 to 3G. I put it away, and spent the next couple weeks thinking how to mount it. Outside would be best, but I live in an apartment on the bottom floor, and the walls outside seem to be made of a concrete of sorts. After further reading of the manual it also seems that I would have to ground the antenna for the extender. Even after all of that, I still would have to drill a whole to bring the coax inside from the antenna to the base, or get a window cable replacement that zBoost sells. Time to hit the manual again and see what my options were.


The manual says that the easiest way to setup is inside near a window. So I tried holding the antenna near a window and did some testing. Success! Good signal, and OK coverage. Had my wife check, and she didn't like seeing the coax hanging off of the curtain rod, so i took it down, but knew what I could do. The next day, I mounted it near the window, ran the coax through conduit I've laid out for my gigabit cabling i did a while back, and voila. All good, AND concealed behind a curtain.


So far, so good. I'm getting good coverage and wrote this blog on my Droid using only Blogaway and a 3G connection. The pics are the antenna mounted (revealed, and concealed) and the base.

Monday, January 18, 2010

Cloning Windows Servers for Free.

So, I was under some crunch time, and needed an imaging solution. Unfortunately, I didn't have access to any paid solutions, so I had to turn to open source. Now, I don't mind open source, but I rarely use it in big production systems. At any rate, I went ahead and chose one and began using it.

Clonezilla seemed to fit the bill. The two servers that needed to be cloned were both HPs. One was an HP Proliant ML350 G5, and the other was a Proliant DL 380 G4 with no optical drive, both using RAID 1. The RAID part wasn't that big of a concern since the image was going to on a single drive eventually, but the first step was to just get an image.

Originally we were looking for a solution that would allow us to image the servers during productions hours, but since no one had the software to do this, and some other obstacles arose, that idea was scrapped. Just for kicks, what I was planning to do if I was pushed to image the server during production hours was do a live image using DD to a USB drive, probably using Cygwin, or a port of DD for Windows. I would have then probably had to have walked the 3rd party tech how to restore from DD on a blank drive, so I'm kinda happy that didn't happen.

So with Clonezilla, I was going to have to image the servers using two techniques. One using the optical drive, the other using a USB drive. Optical was pretty easy. Just burn the latest STABLE ISO to a blank CD, and boot to it. There were two issues I had. The first being that the video was all garbled like a bad antenna signal when TV broadcasts were analog, I just hit Enter twice to see what would happen. I got to the point where it then asked about keyboard config, how I was going to image, etc. Then it started scanning hardware and loading device drivers. During this time it just stopped for about 2 minutes and kind of worried me (no flashing from optical drive, and I couldn't tell if it was hitting RAM or not, then it continued on). To my joy, it was able to see the data just fine.

Using Clonezilla
1. Choose your deployment method: ISO or USB
2. Go through the menus
3. Select the item that says Image
4. Select /dev for partimage
5. THEN plug in your USB device, wait a couple of seconds, and hit ENTER
6. Let it run and you'll have a clone to a image when all is done.

Using Clonezilla on USB
1. Go to here and follow directions: http://clonezilla.org/clonezilla-live/liveusb.php
2. Ensure your server can boot from USB, AND has it chosen to boot from before hitting the disks.
3. Follow steps 2-6 above.

http://clonezilla.org/

Tuesday, June 2, 2009

Using Truecrypt on Acer Aspire One

If you read one of my older posts, then you read that Truecrypt does not support encrypting the entire drive. I set out to see if it was possible just to encrypt the working home drive (I know it can be done, just didn't know how).

Well, a new version of Truecrypt just came out, version 6.1. And guess what? It still doesn't encrypt the whole system drive. Oh well...back to the drawing board.

To upgrade Truecrypt on the Acer Aspire one, follow Macles instructions for the older version, just use the new version of Truecrypt (i.e, replace the names with the newer version).

Great, so you got it going now what, right? Well time to follow these steps.

What you need:
1. A donwload of Truecrypt
2. An external drive or smb/cifs connection (to temporarily move your user directory).
3. Some time on your hands.

Great, now let's get to the good part.

Adjust user login
You want to login as root, this'll make it easier for you to move the user folder. To do this, you'll have to login as root. Here's a nce tip I found that'll allow you to be prompted with a login at boot:
http://snaprails.blogspot.com/2009/01/acer-aspire-one-linpus-password.html

Login as root
After you've made the above adjustements, you should login as root. This'll be self explanatory once you see the prompt.

Move the user folder
This made sense to me because I was already running low on space. Only about 1.7 GB free. My user folder was taking up about 1.4 GB, and I wanted to be able to utilize as much space as possible (keeping in mind to leave a little for the system should I want to modify anything that would take up space on the system.
1. Mount your external drive, or network drive
2. Transfer your user folder to the other drive. (So, copy then delete, or a straight move. I'm superstitious, so I do a copy and then a delete locally)
3. Clean up Trash so that you gain back the free space on your drive.
rm -rf /root/.local/share/Trash/files/*
rm -rf /root/.local/share/Trash/info/*
4. Fire up truecrypt
5. Click on Create Volume, and create a volume and place it where ever you want, as long as "user" has access to it. Choose whatever encryption method you want, and whatever password you want. When it asks for format choose ext2. I had originally chose none, and then was going to format using aufs, but only the /mnt/home directory needs it. The rest of the drive (including /home/user) is actually ext2. I had to then format it after using ext2. Go back to where you created the Truecrypt container (mine is along the lines of /home/encryptedcontainer.tc), and make sure that "user" has access to it:
-rw-rw---- 1 user user 3221225472 2009-06-02 15:05 encryptedcontainer.tc
You can accomplish this with:
chown -R user:user encryptedcontainer.tc
6. Ensure that it's created and that you can mount it.
7. Create a folder called "user" under /home NOTE: Any messages you get just click OK. Ensure that user has access to the folder: chown -R user:user /home/user
7. Once you've mounted your Truecrypt image, go ahead and transfer the data over to it. (Truecrypt will tell you where it's mounted to, usually /media/Truecrypt1)
8. Run the following to put "user" as owner on the contents:
chown -R user:user /media/Truecrypt1 .
NOTE: The "period" after the command. This will ensure that even the .name files will be adjusted (if you put *.* it may include "." and ".." which will effect the directory above it as well).
9. Ensure everything's been trasferred over, and that Truecrypt is good. Now dismount the file in Truecrypt.

Get Truecrypt to Startup and Mount at Boot
1. Now follow the instructions listed here:
http://ubuntuforums.org/archive/index.php/t-1014891.html
2. Save it, and reboot
3. You should now be prompted with 2 authentication windows
a. The first is to authenticate to the Truecrypt container. If you don't authenticate to it, the boot will loop, and you can't get into "user".
b. The second authentication screen is the gdm one which will allow you to actually login to your account.

Once you login, everything should be working fine. The only problem I've exprienced so far was with Jungledisk. I had to remove it, download it, and extract it again. I also had to remove the .junglediskinstance file. The encryption is seemless. You will not see jungledisk running in the taskbar, or top. If you run ps aux | grep jungledisk, you'll see it running where it mounted your container to /home/user.

The advantage of this setup is that:
1. The Acer Apsire One needs authentication now to get into it.
2. The "User" home folder is encrypted now.

The disadvantage of this setup is:
1. It takes a little longer to boot
2. If you forget your password, your SOL, and need to go back to your backup (still got that user folder you copied over?)
3. Someone can still login as root and see where your Truecrypt container is and attempt to brute force it. If the container uses the same password as your Linpus login, then a skilled user could crack your passwd file and use that to crack your password for Truecrypt.

jungledisk + Amazon S3 on Acer Aspire One

I was trying to get this working a while ago, and gave up. I managed to get far enough where I was able to get it to work only using root (sudo, or straight up root). Little did I know I was close to getting it working under "user" but gave up to soon. Eh, it happens...

At any rate, here's how to do it. This assumes that you already have an S3 account configured and downloaded Jungledisk.

1. After you've downloaded jungledisk, extract and untar it.
tar -xvzf jungledisk*
2. check the permissions on the contents. Pay attention to jungledisk and junglediskmonitor, and ensure that they look like below.
-rwxr-xr-x 1 user user 4394430 2009-04-28 11:17 jungledisk
-rwxr-xr-x 1 user user 9164367 2009-04-28 11:17 junglediskmonitor
3. Check /dev/fuse and make sure that the permissons on it are as follows:
crw-rw---- 1 root fuse 10, 229 2008-03-04 08:44 /dev/fuse
4. Next, add "user" to group "fuse" by issuing the following command:
sudo usermod -a -G fuse user
5. Ensure that you have the correct group memebership by issuing the following command, and making sure you see "fuse" in the list:
[user@localhost jungledisk]$ id user
uid=500(user) gid=500(user) groups=500(user),10(wheel),496(fuse)
6. Reboot, otherwise the changes to group membership will not take effect.
7. Go ahead and run junglediskmonitor, once you get back up and running and enter in the requested information.

This is a great addition to my Acer Aspire One using an 8GB SSD!