ZFS, Btrfs and Oracle
Most of you may already be aware of Oracle’s acquisition of Sun Microsystems. If not, here is an article stating just that from Sun’s website. I read the news as soon as it was published on the net along with the reactions of Sun users for Sun products. What will be the future of mySQL or OpenOffice?
It just dawned on me yesterday: What will be the future of ZFS and Btrfs, seeing how Chris Mason, the lead developer to Btrfs works for Oracle? Historically, Btrfs was the Linux community’s solution to ZFS. Sun had intentionally open sourced and released ZFS under the Common Development and Distribution License (CDDL) as a way to prevent it from being integrated into the Linux kernel and allow the Solaris operating system to have a competing lead. Although the license did not stop it from being ported over to the Mac OS X and other BSD based distributions such as FreeBSD (and PC-BSD).
The CDDL conflicts with the General Public License (GPL) version 2. While we saw ports of ZFS on FUSE, this wasn’t something that could allow the kernel to grow and ensure a more dominant future. In order to last in the enterprise class market, the Linux kernel needed a better performing, more feature rich and newer file system. That is when Btrfs stepped on to the stage, which by the release of Linux kernel 2.6.29, it was merged into the kernel tree.
Reading the Btrfs development mailing list, the concern for the future of Btrfs has already been brought up. Chris Mason assures to the team, that Oracle has not changed the plans for Btrfs and that its development and growth will continue. And realistically this would be the way to go. Too much time and money has been invested into Btrfs. To re-license and port a stable version of ZFS into the Linux kernel could take, at a minimum, a couple years. Also, there is this sense of pride within the community. To port ZFS over to Linux could kill that pride, emphasizing that Linux cannot survive on its own and needs to extract its ideas and functionality from elsewhere. I can also see developers and administrators alike, losing respect for Oracle if they were to make such a drastic decision by abandoning all that has been done.
I for one, hope that Oracle stays true to their word. While the ZFS file system is an excellent file system, I know the Linux kernel will be able to compete on its own terms and with its own technologies.
Hi,
I am thinking the same thing, currently using www.nexenta.org for a fileserver as I wanted ZFS and a debian userland.
I dont think you are correct in saying it will take 2 years to port, FreeBSD did it very quickly, see http://wiki.freebsd.org/ZFS (Paweł Dawidek got it kind of working in 10 days)
Tom
Comment by thattommyhall — 17. May 2009 @ 15:48
Tom,
The reason why I said “minimum of 2 years” is that it will most likely take that long to filter out all known bugs and problems and consider the file system stable for storage administration. FreeBSD 7.2 (released May 4, 2009) is the first release to include the port of ZFS and it is still considered experimental. The port of it was started in April of 2007.
Writing a file system with its basic functionality is actually fairly simple. I have written 2 file systems in my day and I can admit that the most difficult part of it is not the initial layout and implementation but instead the chasing of bugs that involve security to even data corruption.
Also, I have been thinking of playing around with Nexenta lately. One of these days I plan to give it a try. Do you have any advise or comments worth noting on this distribution?
Petros
Comment by admin — 17. May 2009 @ 17:16
1. Do you really think that BtrFs will be ready for prime time in less than 2 years from now? ZFS has the advantage that the concepts and the code is under production for quite a few years now. ZFS development started in 2001 and the adoption just got higher for a year or so. BtrFS will need the same time. You can´t compare the development time of ext2/3/4 as those steps as evolutional, not revolutional. BtrFs is something completly new.
2. “While the ZFS file system is an excellent file system, I know the Linux kernel will be able to compete on its own terms and with its own technologies.” is just a nother version of a strong NIH syndrome. There are more important problems to solve than already solved problem.
3. The next point: BtrFs looks similar at the front, but isn´t really similar. Like the problem that disk management and filesystem aren´t integrated. It just looks that way, as a volume manager is controlled out of band. But the advantages of faster resilvering due to the knowledge of the volume manager of the used space can´t be implemented that way.
4. The GPLv2 conflicts with almost all other known Open Source Licenses. This way round it´s more correct.
Comment by Joerg M. — 18. May 2009 @ 00:48
To be fair, 7.0-RELEASE had ZFS, too..
I don’t use FreeBSD or ZFS, but I know there’s some talk of ZFS versions or feature sets. Only 8.0 will have ZFS version 13, with all the bells and whistles on par with current OpenSolaris.
Comment by Alan — 18. May 2009 @ 03:24
Joerg,
You are correct and it will take some time before we will have a stable and trusted build of Btrfs. If a port of ZFS ever occurs, as I mentioned it will also take time to filter out bugs and consider it stable; but there “may also require” (I am leaving room for the slight possibility) some restructering of the Linux kernel to accomodate certain features and/or functionality of ZFS which can impact stability to even performance. Btrfs is built for the Linux kernel and it was from the very beginning. It follows the kernel’s evolution.
Btrfs will take some time to evolve into a fully dependable file system/volume manager. With involvement from Oracle, Red Hat to even independant developers, it may get there much quicker than we think.
I agree with your comment on “There are more important problems to solve than already solved problem.” As feature rich and advanced as ZFS is, at this point, due to the license conflict, there is no other choice and the community needs to respond, even if it involves re-inventing the wheel and writing a new file system.
Alan,
Thank you for the feedback. Even while the 7.x releases with ZFS are considered “experimental” I look forward to playing with it this week in PC-BSD 7.2. I downloaded the ISO and intend on installing it soon.
Petros
Comment by admin — 18. May 2009 @ 09:56
Ahh lunix, reinventing the wheel over and over again..
Comment by James Kay — 18. May 2009 @ 11:12
I know I shouldn’t feed the troll, but James Kay, they’re improving the wheel, not reinventing it.
Comment by Slapo — 18. May 2009 @ 13:21
I’ve been using ZFS since it came out in 7.0 and although it’s experimental it is extremely solid. I have since moved all of my graid3 volumes over to ZFS and have not had a problem. And I don’t think Sun licensed ZFS using the CDDL just to spite linux. Honestly that sounds a bit vain. Almost everything from the Solaris projects are licensed CDDL.
Comment by FBSD User — 18. May 2009 @ 13:31
This fall I’m going to migrate to ext4 and give up caring about file systems until I move to a solid state drive.
Comment by ethana2 — 18. May 2009 @ 15:32
“Too much time and money has been invested into Btrfs”. Have you ever heard of the sunk cost fallacy? You are badly mistaken in thinking that time and money already spent on something is a factor supporting continuation.
“Also, there is this sense of pride within the community”. This would be a variation on the “not invented here” justification.
“To port ZFS over to Linux could kill that pride, emphasizing that Linux cannot survive on its own and needs to extract its ideas and functionality from elsewhere”. Not true. There is something deeply wrong in the Linux community if it cannot recognize that someone outside the community may have come up with an equally good or even better idea and/or implementation of an idea. Do you want a best of breed product or one that is all decked out with pride and “we did it all by ourselves” attitude?
I don’t actually know if ZFS is better than Btrfs or not. What I do know is that your posting raises only one valid issue - the license. But this is the issue that could potentially be addressed by the Oracle acquisition! The correct thing to do here is to recognize opportunity of choice and evaluate the new options based on their technical and practical merits, not logical fallacies and emotions. Maybe Btrfs ends up winning a valid comparison. But maybe ZFS really is the better choice.
BTW, is there any chance you’d consider another font for your postings? This one is making my eyes bleed.
Comment by Logical — 18. May 2009 @ 16:04
There is a more fundamental issue. Linux has JFS and XFS and a number of other externally created filesystems and they aren’t used nearly as much as the ones designed for Linux. They also aren’t supported as much, I ran an all XFS system for a number of years and it wasn’t unusual to install a kernel update or something and have it not able to boot afterwards. Use SELinux? well you might have support or you might not.
The community supports BTRFS, it’s not just more abandonware. Plus it’s always nice to have some competition and choices.
Comment by Frank — 18. May 2009 @ 16:15
The “sense of pride” that the technology in Linux is original that you refer to is ridiculous. I would go so far as to say that most of the concepts implemented in the Linux kernel were developed elsewhere first. The vast majority. The people in the Linux community that actually know what they’re talking about don’t share your delusion.
Comment by Marcus Reid — 18. May 2009 @ 20:11
Marcus,
It is their implementation that makes them original. As was the case with Solaris when it had implemented their zones, dtrace to even ZFS. Sure the Linux kernel has adopted ideas here and there; as has every other OS. Believe it or not but the “delusion” that you allude to is the same “delusion” I hold for Solaris/OpenSolaris which is also the same “delusion” I hold for FreeBSD and so on. The pride that I speak of, is the pride that all the individuals involved (working as a team) on the Btrfs system have. Knowing that they are making a difference. Knowing that they are all striving to achieve the same goal. The goal of something that can bring change. I am sure this same sense of pride was felt by those who developed ZFS. Do we all remember what happened with EVMS when LVM was chosen in its place? I do not know what you do by profession, but this pride can go a long way in determining the quality and support of a product.
Comment by admin — 18. May 2009 @ 21:01
Logical wrote: “BTW, is there any chance you’d consider another font for your postings? This one is making my eyes bleed.”
Navigate to Edit->Preferences->Content. Change the default font and get over it.
Comment by admin — 18. May 2009 @ 21:33
> I am thinking the same thing, currently using www.nexenta.org
> for a fileserver as I wanted ZFS and a debian userland. I dont
> think you are correct in saying it will take 2 years to port,
> FreeBSD did it very quickly, see
> http://wiki.freebsd.org/ZFS (Paweł Dawidek got it kind of
> working in 10 days)
He is correct. There is absolutely no way the Linux developers
are going to bother adding support for a new file system without
putting a lot of effort into integrating it into Linux proper.
This means eliminating code redundancy, getting the coding style
matching up to Linux standards, shoving as much functionality
into the Linux-VFS stuff as much as possible. That sort of thing.
When they integrate features into the kernel it must be done
correctly. _ESPECIALLY_ for file systems. The Linux developers
have been burned before by simply shoehorning driver code into
the kernel without putting a lot of effort into proper
integration.
And with file systems the Linux developers are especially
conservative. Why? Becuase it’s very difficult for people to
upgrade their filesystems. Once you have people using a file
system then their data is going to be stuck there for the longest
time.. you will have a very difficult time being able to
efficiently remove the data from a important system, reformatting
the file system, and then having that data replaced.
——————–
Anyways everybody needs to stop buying into Sun’s bullshit. ZFS
is nice and it has a lot of features that make it desirable, but
it’s not God’s gift to file systems. ZFS is just one of the very few
features that Solaris has over Linux. Dtrace is the other big one…
Other then that, not much.
There WILL be better file systems that come along and I think
that Btrfs is going to be one of them.
Why?
Btrfs supports most of the features that ZFS has and other Linux
file systems do not.
Btrfs is also going to be faster then ZFS. ZFS is nice, but it’s
not a very fast file system when compared to things avialable to
Linux.
There is a clear upgrade path for existing Linux users. Ext2 was
upgradable to Ext3. Ext3 is upgradeable to Ext4. And
Ext3/Ext4 (at least Ext3) will be upgradable to Btrfs.
Btrfs integrates better into the Linux kernel. It’s makes heavy
use of code that is already well tested and used by other file
systems. Improvements to itself often lead to improvements in
other portions of the kernel. ZFS, in comparison, is alien code
from a much different design basis.
Btrfs has optimized modes for solid state hardware. These modes
will not only improve write performance, but improve read
performance, and work to prolong the life of the solid state
hardware.
CRFS is ‘Coherent Remote File System’ for Linux. It is a
distributed file system under heavy development that uses BTFS
features for the daemon and uses BTFS for the on-disk format.
Comment by drag — 19. May 2009 @ 03:28
drag,
You are correct. What is currently in production will remain in production until you can safely convert from the one file format to another (which you mentioned): http://btrfs.wiki.kernel.org/index.php/Conversion_from_Ext3
This is a huge advantage to continue with the development of Btrfs!
Comment by admin — 19. May 2009 @ 10:37
Initially developed by Oracle, Btrfs is licensed under the GPL and open for contribution from anyone!!!
Comment by oraguy — 19. May 2009 @ 15:12
ZFS + lustre looks interesting:
http://arch.lustre.org/index.php?title=Architecture_ZFS_for_Lustre
What about Zones vs VServer?
Comment by Ron — 19. May 2009 @ 20:43
I keep seeing this blog entry when I poll google to see if Oracle’s decided what they’re going to do with zfs, if anything, so I think it’s probably time to interject a thought (not that I’m sure anybody is reading comments on a post this old anymore, but still..)
Has anyone else considered the implication of ZFS gaining widespread support on Solaris, various flavors of BSD, Mac OS X, and Linux? For the first time, it would mean that every operating system being taken seriously except Microsoft’s had a common filesystem with advanced capabilities that each OS understands natively.
Currently the best we have for a common interchange is fat32. Seriously, the best we can come up with for a common denominator is fat32?
Comment by T. Joseph Carter — 9. June 2009 @ 02:22
T. Joseph Carter,
It is very true that ZFS is gaining a significant share of popularity with the non-Windows crowd. And outside of the enterprise arena and into the end-user desktop, Microsoft still holds 87% of the market (numbers from Net Applications). The FAT file systems were so easy to adopt and have set as a commonly supported file system across all operating systems for at least three reasons:
1. Microsoft’s market share
2. Microsoft’s inability to understand the word “interoperability”
3. The usage of removable media preformatted with FAT32.
With that aside, I am sure that there are plenty of more reasons but those seem to be the ones that quickly come to mind.
While ZFS has been creeping its way onto other operating platforms such as BSD, Mac OS X (read-only), and Linux (over FUSE), it is still too bulky of an FS to function in every arena. Please do not misunderstand. I love ZFS and enjoy playing with it. Sun has done an excellent job in designing (and continuing to design) the FS. ZFS with all its features and functions is pretty resource hungry and will not apply to some of the smaller media devices; thus limiting its adoption. FAT32, while inferior, is lightweight which made it extremely easy to port and manage.
I see a strong future for ZFS on Solaris and BSD flavors of UNIX, but not on Linux (due to licensing). At one point in time one could have thought that there was more common ground with UFS on all of the non-Windows platforms, although each distribution has its own customized version catered to their operating systems.
Thank you for your comments.
Petros
Comment by admin — 9. June 2009 @ 07:38
The article says… “Also, there is this sense of pride within the community. To port ZFS over to Linux could kill that pride, emphasizing that Linux cannot survive on its own and needs to extract its ideas and functionality from elsewhere. I can also see developers and administrators alike, losing respect for Oracle if they were to make such a drastic decision by abandoning all that has been done.”
Linux community takes pride for something “Oracle” develops and maintain? And they will lose respect for Oracle if it abandons Btrfs? Why that injustice? It’s unfair!
If Linux community takes pride, then let them prove they deserve it by adopting Btrfs if Oracle abandons Btrfs. That’s justice!
Comment by otan — 16. June 2009 @ 07:45
I have conclusively proven that btrfs is actually a blatant repackaging of reiser4 in a cover up to avoid the political disaster of supporting the code of a convicted murderer. btrfs is 81.56% similar to reiser4. Here are the steps to reproduce. Please spread the word. http://pastebin.com/ff42272d http://pastebin.com/f27912488
Comment by PENIX — 14. July 2009 @ 16:13