dd if=/dev/random of=/dev/blog

After spending a week on the forums at opensolaris.org, it would seem that a great number of individuals are waiting for Oracle to make it official: “that OpenSolaris will be no more.” If this ends up being the case then it would be very upsetting as OpenSolaris is a very stable and robust UNIX operating system. During this time I came across this blog posting which may shed some light. It discusses the Oracle directed Solaris Next label svn_140. Solaris Next is the development name given to Solaris 11.

Initially Oracle had publicly announced that it will stay involved in the OpenSolaris community and continue to support it. This was more than a few months ago, and they have been silent since. If they do decide to kill the OpenSolaris project to shift all focus on Solaris Next alone, then I am curious as to if they will kill the OpenSolaris binary distribution project (opensolaris.com) only or the binary plus the community (opensolaris.org). The community can still live on without the binary. They will instead focus on the Solaris Next builds as opposed to OpenSolaris.

I am still unsure about Oracle’s promise on the future of OpenSolaris. Phoronix has just posted an article showing that what was once supposed to be 2010.02 and then 2010.03 to eventually 2010.1H has not been released at the end of the first half of the year. It would be a shame if nothing came out of this. OpenSolaris is such a great platform and ZFS brought some much needed features which helped increase my development and productivity. Should we cross our fingers for a 2010.2H release?

I guess what it boils down to for Oracle is, do they see a profit from this open source project? What does OpenSolaris bring to Oracle and how does it help them solidify their newly acquired Solaris platform? If we ask for my personal opinion, I feel the advantages outweigh the disadvantages. A working example can be seen with Red Hat and the Fedora Project. The more open platform (in this case OpenSolaris) defines the bleeding edge technologies, stabilizes them and is a testing ground for all that will be ported to the official stable release (Solaris). In fact that is how it was under Sun Microsystems.

Although as the months pass by, the more dim OpenSolaris’ future gets. Fortunately enough it is an open source project and can easily be forked into a new community driven distribution. So even in the darkest of hours not all is lost.

Earlier this week I came across this blog posting about data integrity testing on ZFS title: ZFS data integrity tested. It was a few months old from Robin Harris’ blog Storage Bits. I guess the most exciting part was validating Sun Microsystem’s claims to ZFS having the ability to correct data corruption even with error injection to both the disk and memory. ZFS continues to prove its worth on enterprise class systems and applications.

My only frustatrions with ZFS are that cluster support is currently not available, at least until Lustre 3.0 is out, whenever that will be. Another frustration is trying to write an application that will work directly with a zpool. For instance, there is no simple method to send a zpool a generic ioctl() such as DKIOCGGEOM to obtain the size of the volume. In most cases I don’t care about the number of cylinders, heads and sectors. In the end I calculate the total volume block and/or byte count. So those values could be generic and made up.

In the early stages of my discovering this, I posted a simple question on the OpenSolaris Forums:

“As I was navigating through the source code for the ZFS file system I saw that in zvol.c where the ioctls are defined, if a program sends a DKIOCGGEOM or DKIOCDVTOV, an ENOTSUP (Error Not Supported) is returned.

You can view this here: http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/fs/zfs/zvol.c

My question is, what is an appropriate method to obtain the zpool’s volume size from a C coded application?”

After posting my question, I immediately went to view the open source to the general zpool/zfs binaries and observe how zpool reported the drive pool’s capacity back into user space. Unfortunately it utilized some cryptic method not as straight forward as sending a simple ioctl() to the desired volume. This was a bit frustrating as it was such an ugly approach to only receive the size of the volume.

I was grateful to have a response confirming my fear of choosing the ugly route; but it also made me realize the true value of open source. What if I simply patched a supported ioctl() definition to return the total accessible “block” count of a zpool? It would be similar to the Linux BLKGETSZ/BLKGETSZ64. This would be the most realistic and proper method; to add a new ioctl() and then modify all storage modules to accommodate it. For instance in the usr/src/uts/common/sys/dkio.h file we would need to define:

And then go back to the zvol.c file and add the extra ioctl() to handle this:

To give a level of consistency across all storage devices, we will need to add the ioctl() definition to the following modules:

Although we do not necessarily have to support it and can instead interpret it as such:

Who knows, one of these days I may get around to patching this myself and if the OpenSolaris community doesn’t accept it I can always make it available on any one of my website. I will most definitely post about it.

As long as I can remember I had been using the Netscape web browser which evolved to Mozilla and now Firefox. I still use Firefox and have grown so comfortable with it that I don’t really desire to move onto anything else. Needless to say, Mozilla’s products are not perfect and there is always room for additional features and what I believe to be necessities in order to function in today’s world of computing.

For instance, I wish there was more of a concentrated effort to bring additional usability and manageability of these same Mozilla products (i.e. Firefox, Thunderbird, to even the SeaMonkey suite) where it can truly compete with Microsoft to even IBM’s Lotus in the professional world.

In this initial example I will choose to focus Lotus Notes and SameTime. This application was built around enterprise productivity. Everything is integrated in such a way where I do not only have the ability to work from my e-mail but through SameTime I can easily connect to the same directory of individuals for instant IM messaging. Google saw an advantage to such an approach and had it integrated into their GMAIL web client. So how difficult would it be to have Thunderbird do the same thing? Now, SeaMonkey may be a better candidate for this integration (as it resembles more of what Netscape used to be) but nobody really knows of its existence.

Another example is the way Microsoft integrates all of its products together to provide a complete solution. If I am using Internet Explorer and I click on something that requires Powerpoint, Excel or something else Microsoft developed, there are no problems in opening up those files and working with them in a new tab of my web browser. It could be beneficial for Mozilla to partner up with Oracle and provide similar transparent integration into their products with Star Office and OpenOffice.org. In fact, what is stopping Red Hat, Canonical, Novell or even Oracle from developing such integration modules in their workstation solutions? I always found it annoying that when I click on a PDF file to open it up in a new tab, it runs outside of the browser instead (unless I were to install a third party developed plug-in most of which are written for Microsoft Windows anyways).

Other things that I would like to see Mozilla work on is better management for corporate environments. Internet Explorer has a centralized managed infrastructure in that it can be controlled using Group Policies across an entire network emphasizing consistency. If you need to change the settings across all web browsers in your network, IE makes that easy and realistic. It is not realistic to e-mail co-worker HOWTOs (especially if they do not have the required permissions) nor is it realistic to visit or remote into every node to address those same changes. This form of management also includes patching/upgrading the browser/e-mail clients. Sometimes this needs to be controlled on a corporate basis as opposed to an individual user.

Mozilla has been doing an excellent job in conquering their fair percentage of market share. All without the billions of dollars dumped into marketing (as seen by their competitors). They are a known household name. A lot of end users know what Firefox is. To move on to the next best thing, I feel they need to start concentrating more on the corporate world.

It would appear that the OpenSolaris project is back on track. This is a bit long overdue. The acquisition of Sun by Oracle left a few projects in questionable states. It was unknown as to whether Oracle would continue supporting these open source projects. OpenSolaris was included in that list.

The great news is, Oracle locked down b134 (which can be obtained at Genunix). It is going to be build for internal testing and we are possibly looking at a May release. I sure do hope they clarify the release name and rename it from 2010.03 to 2010.05.

All of this can be read here. It was Albert Lee who posted the following:

“Those of us feeling left in the dark might be pleased to know that build
134a, the first candidate for the next stable release of OpenSolaris, has
been tagged in Oracle’s release branch (in project jargon: “snv_134a, the
first respin of 134, closed earlier this week”). A packaged build should be
available for internal QA soon, but even if it passes, it will be while
some time before the release can be published to the external repo.“

Two days ago, I ran through a long needed image update to the OpenSolaris 2010.03 preview. I was updating through the pkg update manager from build 129 to build 134. So when I say, it was much needed, I wasn’t kidding. Anyway, after over 1 GBytes of updates was completed, a new boot environment (BE) was created with the native ZFS snapshot feature and I shut down the PC for the night.

The next day I turned the PC on into the latest boot environment to find that my gnome-terminal was giving me problems. The obvious symptom was that certain characters were not being echoed and their was misalignment with every entry and output displayed within the terminal.

After some research I came across OpenSolaris bug 12380: image-update loses /dev/ptmx from /etc/minor_perm. The fix (workaround) was simple: boot into the previous boot environment, mount the newest boot environment and clone the /etc/minor_perm from the one to the other. The steps are as follows:

And the problem was fixed. It was quick and easy thanks to ZFS.

This entry is a continuation of one published in May of 2009. In fact it is relating to a comment made earlier today which I responded to in brief words. I am now taking the time to offer my viewpoint on the whole ZFS licensing under the CDDL and the reasoning for it.

It wasn’t until I started working with the OpenSolaris kernel and by working I mean, modifying code and going through the build process that I finally realized why OpenSolaris was licensed under the Common Development and Distribution License (CDDL). A lot of other people and companies have claim to code used within Solaris. That includes copyrighted code to which Sun does not have the authority to publish in an open source license. This is why they needed to work with a weak copyleft license such as the Mozilla Public License and modify it to their expectations. The CDDL was eventually approved by the Open Source Initiative (OSI) as a valid open source license and Sun Microsystems was then able to release code under its limitations.

Now before I continue I wish to describe 3 different open source licensing models: (1) the strong copyleft license, (2) the weak copyleft license and (3) the non-copyleft license.

The strong copyleft license is a project based license in which it requires that any derived code from the original project must remain under the original license. This method of licensing makes it nearly impossible to link with code under a non-strong copyleft license. As a result of this approach, strong copyleft licenses are often referred to as viral licenses. The most popular of these licenses is the General Public License (GPL) with 3 available versions. The Linux kernel is licensed under this and its success and growth can be attributed to it.

The weak copyleft license is similar to the strong copyleft license except that it is file-based instead of project based. This means that if there are any modifications to a file, the original license must apply; but that file can be combined in a project with code under a different license. This method makes the type of licensing non-viral. The CDDL and the MPL are categorized as weak copyleft licenses.

The third type is the non-copyleft license which offers no requirement for derived works to stay under the original license. In fact, there is also no requirement for derived code to be released under any open source license. This makes it simple for someone to take an open source project and use it as a basis for a proprietary product. A best known example is the BSD license; and Apple’s adoption of FreeBSD kernel code in their XNU kernel or NetApp and their use of FreeBSD in their customized storage appliances.

Continuing where I left off, it would not have been possible to open source the Solaris kernel for the OpenSolaris project if it weren’t for the CDDL license. In turn, ZFS would have been incompatible with the CDDL license if it were licensed under the GPL; although it has no conflict with non-copyleft licenses such as the BSD license. Because of this and now because of Oracle’s admitted support and commitment to Solaris, I doubt this licensing will change; especially to merge it into the Linux kernel. That is why we should be grateful that: (a) ZFS is available under an open source license making it impossible for it to disappear and (b) that Oracle has been committed to Btrfs and bringing an enterprise class solution into the Linux kernel.

This is why we have choices. If you want ZFS functionality, use OpenSolaris or Solaris. If you don’t necessarily need ZFS and are more comfortable with Linux, you have a lot more distributions to choose from. Or if you want ZFS and a familiar Linux environment, there is also Nexenta.

Well, it has been 23 days or so since my last update. Things have been a bit hectic. For instance, I am spending all of my free time in a start-up company intended to cater to the data storage industry. My business partner and I have a few commitments with a couple of technology partners to deliver a data storage management suite. Other time is being spent in writing a book for No Starch Press on OpenSolaris. The book is outlined for 12 chapters and the first 3 have already been submitted to the publisher.

Some other exciting stuff taking up free time is that my wife is expecting our first born this January. Between now and then, our baby daughter can show up into this world, so we are on high alert. Although so far, all is looking well.

I have also spent some time playing around with Ubuntu 9.10 Karmic Koala. It had been installed on my wife’s laptop and we have yet to see any problems or concerns. It was a brand new Sony Vaio laptop and all hardware was recognized without an issue. The operating system runs extremely quick and very smooth. Note that she is coming from the LTS Hardy Heron release, so there are some noticeable changes to her.

I must admit that Canonical is doing a great job with this distribution and I look forward to the next LTS release. I am even thinking of making a permanent switch from Fedora Linux to Ubuntu Linux. Please do not misunderstand me. I enjoy Fedora and have always been a fan. The thing with Fedora is that it is a bleeding edge technology distribution and things have a tendency to break every now and then. I have found myself with less time to go in and address those issues. Although it will not stop me from running a stable installation of RHAS on my Intel servers.

Going back to Ubuntu 9.10, while I find the integration of Plymouth a bit redundant in splash screens (one traditional and one in X; the second after an early initialization into X), one thing that I am really impressed and intrigued by is the integration of Upstart. Upstart reminds me so much of the Service Management Facility (SMF) found in Solaris/OpenSolaris as it shares some of the same basic functionality. It is Upstart that helps to speed up the boot process while also offering a nice and uniform service manager to replace the traditional init daemon. It is just amazing to see my wife’s laptop get to the desktop within 25 seconds from post. Some other positives I saw were in the addition of the Ubuntu Software Center and Ubuntu One cloud-based storage service.

Most of my recent computing hours have been spent in OpenSolaris anyways. Most of it is in development of the earlier mentioned application suite while also using it for the OpenSolaris book. The 2010.02 release is really looking good as I have been playing with build 128. It is currently installed on my Asus Eee 901 and also running on one of my 1U Sun Fire Intel servers. I even took the time to really tune the OS on the netbook and it is running fairly well. That includes, disabling unnecessary services, customizing the CPU configuration (modify the cpupm option to read the following in the /etc/power.conf file: cpupm enable poll-mode) file to utilize less power, enable ZFS compression and disabling ZFS atime updates. I even took the time to enable a RAM-based mounted file system for Firefox caching.

Nothing else new to really report but I am looking forward to the coming year and the future of some of these open source projects. Part of that is the excitement is Linux in the mobile computing industry. Linux had always had a good market share in the mobile industry. There just seems to be more excitement around Google’s Android and in turn Chrome OS.

I just stumbled onto this blog entry on the implementation of data deduplication into the Sun Microsystem’s ZFS file system. It is implemented in such a nice and clean way, I am looking forward to testing it. For instance, just like any other feature of the ZFS file system, data dedup can be enabled disabled at any path from the ZFS root mount point. Examples taken from Jeff Bonwick’s blog post cited above:

It is that simple (man 1 zfs).

On 23 October, 2009 it was announced on MacOSForge that Apple had decided to discontinue any and all development on the porting of the ZFS file system. I know that I am not the only one to say this but I am not surprised. Supposedly there were legal reasons behind this action but in the end, who cares? They are the ones losing out to continue with an out dated and still limiting file system.

Now Apple has recently been hiring file system developers to develop a next generation file system to replace the traditional HFS+ but (as Robin Harris has previously stated) how long will it take before it becomes stable and accepted by the general public? Traditionally it takes 5+ years before a file system is considered somewhat stable and ready for production use. It wasn’t until recently that ZFS was starting to make its impact in the enterprise scene. Though my question is, to whom will this next generation file system cater to? I am to assume that it will be for the general end user utilizing Mac devices that “don’t require the weight of the ZFS features and functionality” ; or so it has been said regarding the topic of Apple abandoning the ZFS project. If that is the case and is the primary focus of the new file system, how will this impact their server market share? We already know that there is no such thing as a perfect file system that will perform ideally in every arena it is thrown into. Some will excel more than others and is entirely dependent on its implementation and workload.

In past posts, I have always stressed the importance of the file system and what is integrated within the file system. I routinely point out the numerous drawbacks and limitations of the NTFS driver. Sure, Microsoft compensates for the “lack of features” with applications, services and additional APIs to fill in all those gaps. A good example is VSS (shadow copy). This can impact performance as it is taking file system concepts out from kernel mode and into user land and consuming user mode resources. All these feature should and need to be incorporated into the file system driver. That way we can ensure that there is stability and consistency with all functions the file system performs. Even the general layout is not ideal for traditional computing over large storage media; as the fragmented large seeks between the MFT and the file data can put a lot of stress on the magnetic device. Going back to HFS+ and sort of on the same topic (although the concept is a bit different), the same could be said about Apple’s Time Machine and it running as an application on top of the driver.

One thing that I hold to heart when it comes to file systems is the ability and flexibility to tune it even without taking the mounted device(s) off-line. Most modern UNIX and Linux file systems offer a lot of tunable features (built into the driver!). For instance (through the ZFS character device node) I can dynamically alter file system variables (man 1 zfs). For this example I will focus on access times. Let us say I am using an SSD and decide that it would be more cell friendly and better performing to disable file access times on the root mount.

To view current settings and disable this feature you would type the following in the command-line terminal:

I just hope that Apple is prepared for the journey they are about to embark on. They obviously have file system development experience, and I have no doubts that they have the talent. Do they have the patience and time to invest?