dd if=/dev/random of=/dev/blog

Well, it has been 23 days or so since my last update. Things have been a bit hectic. For instance, I am spending all of my free time in a start-up company intended to cater to the data storage industry. My business partner and I have a few commitments with a couple of technology partners to deliver a data storage management suite. Other time is being spent in writing a book for No Starch Press on OpenSolaris. The book is outlined for 12 chapters and the first 3 have already been submitted to the publisher.

Some other exciting stuff taking up free time is that my wife is expecting our first born this January. Between now and then, our baby daughter can show up into this world, so we are on high alert. Although so far, all is looking well.

I have also spent some time playing around with Ubuntu 9.10 Karmic Koala. It had been installed on my wife’s laptop and we have yet to see any problems or concerns. It was a brand new Sony Vaio laptop and all hardware was recognized without an issue. The operating system runs extremely quick and very smooth. Note that she is coming from the LTS Hardy Heron release, so there are some noticeable changes to her.

I must admit that Canonical is doing a great job with this distribution and I look forward to the next LTS release. I am even thinking of making a permanent switch from Fedora Linux to Ubuntu Linux. Please do not misunderstand me. I enjoy Fedora and have always been a fan. The thing with Fedora is that it is a bleeding edge technology distribution and things have a tendency to break every now and then. I have found myself with less time to go in and address those issues. Although it will not stop me from running a stable installation of RHAS on my Intel servers.

Going back to Ubuntu 9.10, while I find the integration of Plymouth a bit redundant in splash screens (one traditional and one in X; the second after an early initialization into X), one thing that I am really impressed and intrigued by is the integration of Upstart. Upstart reminds me so much of the Service Management Facility (SMF) found in Solaris/OpenSolaris as it shares some of the same basic functionality. It is Upstart that helps to speed up the boot process while also offering a nice and uniform service manager to replace the traditional init daemon. It is just amazing to see my wife’s laptop get to the desktop within 25 seconds from post. Some other positives I saw were in the addition of the Ubuntu Software Center and Ubuntu One cloud-based storage service.

Most of my recent computing hours have been spent in OpenSolaris anyways. Most of it is in development of the earlier mentioned application suite while also using it for the OpenSolaris book. The 2010.02 release is really looking good as I have been playing with build 128. It is currently installed on my Asus Eee 901 and also running on one of my 1U Sun Fire Intel servers. I even took the time to really tune the OS on the netbook and it is running fairly well. That includes, disabling unnecessary services, customizing the CPU configuration (modify the cpupm option to read the following in the /etc/power.conf file: cpupm enable poll-mode) file to utilize less power, enable ZFS compression and disabling ZFS atime updates. I even took the time to enable a RAM-based mounted file system for Firefox caching.

Nothing else new to really report but I am looking forward to the coming year and the future of some of these open source projects. Part of that is the excitement is Linux in the mobile computing industry. Linux had always had a good market share in the mobile industry. There just seems to be more excitement around Google’s Android and in turn Chrome OS.

Earlier today I came across this interesting article on tuning your SSD drive to achieve greater performance. It is worth noting that this article is intended for Linux and when it mentions setting your file systems mount options with noatime, this too is relevant for file systems that support such an option.

I would also take the time to read the comments. There are some distribution specific responses to the author’s notes.

I just stumbled onto this blog entry on the implementation of data deduplication into the Sun Microsystem’s ZFS file system. It is implemented in such a nice and clean way, I am looking forward to testing it. For instance, just like any other feature of the ZFS file system, data dedup can be enabled disabled at any path from the ZFS root mount point. Examples taken from Jeff Bonwick’s blog post cited above:

It is that simple (man 1 zfs).

On 23 October, 2009 it was announced on MacOSForge that Apple had decided to discontinue any and all development on the porting of the ZFS file system. I know that I am not the only one to say this but I am not surprised. Supposedly there were legal reasons behind this action but in the end, who cares? They are the ones losing out to continue with an out dated and still limiting file system.

Now Apple has recently been hiring file system developers to develop a next generation file system to replace the traditional HFS+ but (as Robin Harris has previously stated) how long will it take before it becomes stable and accepted by the general public? Traditionally it takes 5+ years before a file system is considered somewhat stable and ready for production use. It wasn’t until recently that ZFS was starting to make its impact in the enterprise scene. Though my question is, to whom will this next generation file system cater to? I am to assume that it will be for the general end user utilizing Mac devices that “don’t require the weight of the ZFS features and functionality” ; or so it has been said regarding the topic of Apple abandoning the ZFS project. If that is the case and is the primary focus of the new file system, how will this impact their server market share? We already know that there is no such thing as a perfect file system that will perform ideally in every arena it is thrown into. Some will excel more than others and is entirely dependent on its implementation and workload.

In past posts, I have always stressed the importance of the file system and what is integrated within the file system. I routinely point out the numerous drawbacks and limitations of the NTFS driver. Sure, Microsoft compensates for the “lack of features” with applications, services and additional APIs to fill in all those gaps. A good example is VSS (shadow copy). This can impact performance as it is taking file system concepts out from kernel mode and into user land and consuming user mode resources. All these feature should and need to be incorporated into the file system driver. That way we can ensure that there is stability and consistency with all functions the file system performs. Even the general layout is not ideal for traditional computing over large storage media; as the fragmented large seeks between the MFT and the file data can put a lot of stress on the magnetic device. Going back to HFS+ and sort of on the same topic (although the concept is a bit different), the same could be said about Apple’s Time Machine and it running as an application on top of the driver.

One thing that I hold to heart when it comes to file systems is the ability and flexibility to tune it even without taking the mounted device(s) off-line. Most modern UNIX and Linux file systems offer a lot of tunable features (built into the driver!). For instance (through the ZFS character device node) I can dynamically alter file system variables (man 1 zfs). For this example I will focus on access times. Let us say I am using an SSD and decide that it would be more cell friendly and better performing to disable file access times on the root mount.

To view current settings and disable this feature you would type the following in the command-line terminal:

I just hope that Apple is prepared for the journey they are about to embark on. They obviously have file system development experience, and I have no doubts that they have the talent. Do they have the patience and time to invest?

Linked off of www.linuxleak.com, today I found this interesting article on “Linux Tuning The VM (memory) Subsystem.” The author also offers some suggestions for a more efficient computing environment.

According to the H Open Source website, Sun just recently announced the availability of Solaris 10 Update 10/9.

Read more here and here. You can download the OS release here.

Linked from linuxtoday.com, I found an interesting article posted on FlexTk regarding NAS Performance Comparisons between Linux, Windows and OpenSolaris. The results are very interesting. Under each category, comparisons are drawn between:

• Red Hat Enterprise Linux 5.3 (64-bit)
• Ubuntu Server 9.04 (64-bit)
• OpenSolaris 2009.06 (64-bit)
• Windows Server 2003 (64-bit)
• Windows Server 2008 (64-bit)
• Windows Storage Server 2008 (64-bit)

I assume that each operating system is utilizing the default file systems with default settings for that specific release. Red Hat and Ubuntu should be using Ext3-fs, Windows obviously uses NTFS while OpenSolaris is built on top of ZFS. The CIFS/NFS exported share(s) in turn are running on top of these defaulted file systems. Either way, with average overall performance, OpenSolaris seemed to really shine. It also did well in some of the other categories which made sense when knowing the design of the ZFS file system.

Earlier this month Linux+ Magazine released their 4th quarter issue (4/2009) containing my article on The Linux RAM Disk. You can find my article on Data Backup for small-to-medium sized business environments in the next issue: 1/2010.

Yesterday I came across this excellent article on log-structured file systems and their implementation on SSD technologies. It is worth the read.

A few days ago I received the latest issue of Linux Journal Magazine. I must admit that one of the sections I look forward to reading is diff -u. This section summarizes the latest updates and discussions of the Linux kernel development community. It becomes much easier to read a summary as opposed to signing up for the mailing list because you will just get bombarded with e-mails which can be overwhelming the majority of the time.

While reading I came across a Montavista developed project called pramfs. In summary pramfs is a non-volatile RAM based file system, similar to your ramfs and tmpfs with a few differences to distinguish it from the others and in turn adapted for an embedded environment. Two obvious differences are that it is persistent like a traditional disk-based file system and does not reside in volatile DRAM. Pramfs is not new. It was originally announced back in 2004. It is designed to be a simplified file system that does not carry the same weight of the journal-based file systems.

Apparently there had been some problems with the patch being merged into the Linux kernel for a number of reasons. (1) Montavista was attempting to patent some of the concepts and algorithms used in the file system (in 2004) and (2) even after the dropped the idea of patenting their code, there was some discussion on the redundancy of having yet another file system implemented into the Linux kernel (in 2009). What that means, is that the Linux kernel already has two commonly used RAM file systems and a large number of other file systems. So why was there a need to write another one? Why couldn’t Montavista patch already existing code? (3) It is also not a full featured file system in that it does not support symbolic links.

I agree with this logic. Please do not misunderstand me. Montavista is a very respectable company that has done an excellent job in supporting embedded Linux. I am also glad to see them contribute to the kernel and in turn the community. But truth be told, tmpfs was build on top of the ramfs code. Why couldn’t pramfs follow the same course of development. The GPL makes it easy to not have to re-invent the wheel.

The two most noteworthy goals achieved for pramfs (1) is to work with NVRAM and (2) provide and interface that does not utilize the kernel page caching mechanism. By utilizing the DIRECTIO flag available in the 2.6 kernel, Montavista claims that I/O performance is increased significantly to an already high performing interface. Pramfs also allows the user to specify regions of memory for file system usage.

With it working in non-volatile memory, the data contents will remain intact even after an expected/unexpected power cycle.

This concept got me thinking a bit. How difficult would it be to add some of these features in Ramfs? Ramfs offer some similar functionality as in it does not use the kernel’s page cache for file I/O.  Tmpfs was designed to offer that functionality along with additional file system control and limitations. Ramfs also has a slightly similar general file system layout. Sure a few structures and routines need to be redefined but that isn’t a big deal in the grand scheme of things.

I mention this in the light of some of the latest headlines circulating through the internet regarding Linux Torvalds’ comments on the kernel being bloated. Does the kernel leave room for additional “bloat” or would it be wiser to add on top of current features/functionality? I would love to read some of your opinions.

For more blog posts relating to RAM-based file systems and RAM Disk device drivers, you can find them posted here, here and here.