Quantcast

BtrFS Usage/Complaints

classic Classic list List threaded Threaded
54 messages Options
123
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

BtrFS Usage/Complaints

Jeff Mahoney
Hi all -

Over the past year, btrfs has received substantial attention in the
areas of both stability and performance. I've been running it,
personally, on a 4 TB data volume as well as my root partitions for my
openSUSE 12.2/12.3 systems. I've been able to do stress tests like a
long fsmark run simultaneously with creating and removing snapshots
(with syncs to force them to disk) and haven't run into any issues.
David Sterba, who's been leading the charge at SUSE for btrfs stability
has likewise regularly run it through an array of torture tests and has
found that problems are occurring a whole lot less frequently than they
have in the past. The upstream btrfs community in which we participate
has also started focusing less on feature development and more on
stability and performance.

But these are only anecdotal data points from two file system guys and
if my experiences working for a major operating systems vendor have
taught me anything, it's that our users can be a lot more creative at
finding ways to break things than we are. ;)

So, I'd like to hear your stories. What's worked for you? What hasn't
worked? What would you consider the pain points with using btrfs?

Lastly, I'd like to ask that you take the opportunity to test with the
latest openSUSE Factory kernel running 3.10-final. 12.3 used the 3.7
kernel since it was the most recent release going into the beta cycle
and we've seen a bunch of btrfs fixes since that release.

I look forward to your feedback and the opportunity to improve btrfs for
the 13.1 release and future releases.

Thanks!

-Jeff

--
Jeff Mahoney



--
Jeff Mahoney
SUSE Labs


signature.asc (858 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BtrFS Usage/Complaints

Claudio Freire
On Thu, Jul 4, 2013 at 12:54 PM, Jeff Mahoney <[hidden email]> wrote:
> So, I'd like to hear your stories. What's worked for you? What hasn't
> worked? What would you consider the pain points with using btrfs?

Last I heard, btrfs had no provision for reserved space. This means,
once a btrfs partition becomes full, it may become rather difficult to
fix[0]: deleting can't succeed if there really isn't any space left.

That really needs fixing.

[0] https://btrfs.wiki.kernel.org/index.php/FAQ#Help.21_Btrfs_claims_I.27m_out_of_space.2C_but_it_looks_like_I_should_have_lots_left.21
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BtrFS Usage/Complaints

Graham Anderson-3
In reply to this post by Jeff Mahoney
On Thursday 04 Jul 2013 11:54:05 Jeff Mahoney wrote:
> So, I'd like to hear your stories. What's worked for you? What hasn't
> worked? What would you consider the pain points with using btrfs?

On a several desktops and two small hosts and several VMs with limited size
(20GB) root volumes, I had to disable/remove snapper and manually delete all
the snapper created sub volumes. Without warning, the root volumes became un-
writeable with write operations from various services and applications
reporting no available space.

I understand why df/du and monitoring tools are unable to correctly report
free space in the same way that we are used to with more traditional non COW
filesystems; but unless we educate users/admins about "btrfs filesystem df" or
improve the various traditional reporting tools I bet the problem is going to
bite more people.

I didn't investigate the problem further to determine whether it was a lack of
space for meta information/extents or whether the sheer number of subvolumes
(even with COW) was taking up the space.

The problem generally manifested itself after a medium period of about 3-4
months use, but on one desktop and one VM which were heavily used for testing
packages, the problem appeared within a few weeks. Not good at all...

I'm concerned that smallish sized root volumes may be quite a common choice
for VM's and desktops and that this problem might appear more often as
adoption increases.

The second issue I ran into was a while ago and I apologise for no bug report
on this one (I had no clue if btrfs was even officially supported in yast at the
time). I can't remember the specifics but I was manually creating a btrfs
filesystem with either multiple physical disks or md devices. The system
partitioner (yast) was unable to recognise the filesystem; but what's worse is
that I was able to add a btrfs filesystem via yast to the same physical or md
devices that I had already manually created an fs on.

In general, btrfs support in yast appears fairly limited and I would imagine
I'd get really pissed off in future if the only supported brtfs use was via
yast and that didn't allow for what I imagine will be fairly popular btrfs
usage (multiple physical devices for large storage system). So I guess what
I'm saying is I'd love to see some more love for btrfs in YAST with better
support for btrfs options ;)


Cheers the noo,
Graham


--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BtrFS Usage/Complaints

Yamaban
In reply to this post by Jeff Mahoney
On Thu, 4 Jul 2013 17:54, Jeff Mahoney <jeffm@...> wrote:
[snip]
> So, I'd like to hear your stories. What's worked for you? What hasn't
> worked? What would you consider the pain points with using btrfs?

Using OSS 12.3 (out of the box) it was not possible to use btrfs for
/boot, most boot-loaders disliked that something firce.

Has that changed? What's the prognosis?
(For lilo, grub-legacy, grub2, gummi-boot, syslinux, ...)

Otherwise, I'd like to say fine so far, but I did not have any
disaster so far, too.

The snapper tools could use some love, esp. the filters:
excluding files / dirs would be very nice,  expanding the
man-page with some more examples (or adding a man snapper-config)

Thanks so far, for a desktop using btrfs for /home with use of
snapper has helped to reduce some panic-attacks.

On performance, we could not 'feel' any difference between ext4
and btrfs, the difference between ssd and hdd is much easier to feel.

The missing implemention of a reserved space is much more critical.
(create a small btrfs partition, fill it, then try to delete some,
that is NOT working, you have to expand the partition and the fs first)

  - Yamaban.
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BtrFS Usage/Complaints

Jan Engelhardt-4
In reply to this post by Jeff Mahoney

On Thursday 2013-07-04 17:54, Jeff Mahoney wrote:
>
>So, I'd like to hear your stories. What's worked for you? What hasn't
>worked? What would you consider the pain points with using btrfs?

Using `rsync --inplace` to update files that have shared extents
(like, in snapshots) leads to what seems to be terrible
fragmentation — so much that said rsync command progresses at
considerably reduced speed, noticable when updating "big" files where
there is a chance to look at the rsync -P progress.

I know it is a tradeoff, and I am not sure there is actually any
fits-it-all solution. I know I could drop the --inplace argument, at
the cost of spending more space. The bigger the file and the lesser
the amount of data changed per snapshot, the costlier this will be.
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BtrFS Usage/Complaints

Graham Anderson-3
In reply to this post by Jeff Mahoney
On Thursday 04 Jul 2013 11:54:05 Jeff Mahoney wrote:
> So, I'd like to hear your stories. What's worked for you? What hasn't
> worked? What would you consider the pain points with using btrfs?

When I tested a variety of KVM image formats and settings to see what worked
well with regards to disk read/write speed; I found that any disk image stored
on a btrfs volume made writing to the the VM filesystem (ext4) too slow to be
usable for any kind of real work load.

I didn't test using btrfs volumes inside the VM's themselves.


Cheers the noo,
Graham

--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BtrFS Usage/Complaints

Jan Engelhardt-4
In reply to this post by Yamaban

On Thursday 2013-07-04 19:21, Yamaban wrote:
> On Thu, 4 Jul 2013 17:54, Jeff Mahoney <jeffm@...> wrote:
> [snip]
>> So, I'd like to hear your stories. What's worked for you? What hasn't
>> worked? What would you consider the pain points with using btrfs?
>
> Using OSS 12.3 (out of the box) it was not possible to use btrfs for
> /boot, most boot-loaders disliked that something firce.
> Has that changed? What's the prognosis?

You need a boot loader that understands the btrfs layout.
IIRC, that's the behemoth called grub2.

But then again, I have arranged myself with having /boot on some
"simple" filesystem, also because the delayed writeout, or internal
rebalancing of some filesystems (and maybe even manual
defragmentation!) means that the location of stage2 might change,
which is totally uncool for bootloaders that grab stage2 solely by
LBA numbers.

--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BtrFS Usage/Complaints

Jan Engelhardt-4
In reply to this post by Graham Anderson-3
On Thursday 2013-07-04 19:26, Graham Anderson wrote:

>On Thursday 04 Jul 2013 11:54:05 Jeff Mahoney wrote:
>> So, I'd like to hear your stories. What's worked for you? What hasn't
>> worked? What would you consider the pain points with using btrfs?
>
>When I tested a variety of KVM image formats and settings to see what worked
>well with regards to disk read/write speed; I found that any disk image stored
>on a btrfs volume made writing to the the VM filesystem (ext4) too slow to be
>usable for any kind of real work load.

Yes, that's the snapshot-of-inplace-updated-files issue I too described.
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BtrFS Usage/Complaints

Jon Nelson-5
In reply to this post by Jeff Mahoney
On Thu, Jul 4, 2013 at 10:54 AM, Jeff Mahoney <[hidden email]> wrote:
> Hi all -
>
> So, I'd like to hear your stories. What's worked for you? What hasn't
> worked? What would you consider the pain points with using btrfs?

I'm using openSUSE 12.3 with and without more recent kernels.
On one 4-disk btrfs filesystem, I ran into filesystem corruption that
- even though the upstream btrfs folks have been working on it - is
not yet resolved. I was (and am) able to mount the filesystem with "-o
ro,recovery" but that's it.

I use btrfs on 3 out of 5 computers running openSUSE. I've had some
really amazing positive experiences with it. In one case, I had a
4-disk btrfs filesystem where I had accidentally bumped the sata
cable, just enough that it would kinda-sorta work but threw errors
left and right. It accrued over 90 thousand errors before I noticed.
After reseating the cable and running a "scrub", all 90 thousand
errors were repaired.

On my (only) SSD, it's been pretty solid. Unfortunately, all 3.8, 3.9
kernels lock up almost instantly when a file is "defrag"d. A
combination of two patches from upstream appears to resolve that for
me, but those patches are not in the stock openSUSE or vanilla
kernels.

Overall, it's been positive with some performance (fragmentation)
issues.  I filed a number of bugs in the openSUSE bug tracker, some of
which have gone unacknowledged and all of which remain open,
unfortunately. Please do not take that as an indictment of the huge
quantity of work, effort, and expertise that a great many persons put
into openSUSE - I recommend no other distribution!

> Lastly, I'd like to ask that you take the opportunity to test with the
> latest openSUSE Factory kernel running 3.10-final. 12.3 used the 3.7
> kernel since it was the most recent release going into the beta cycle
> and we've seen a bunch of btrfs fixes since that release.

I'll give it a try as soon as it's available. Are you talking about
the OBS project with title:
"Kernel builds for branch stable (standard)" ?

> I look forward to your feedback and the opportunity to improve btrfs for
> the 13.1 release and future releases.

13.1 is looking very nice!


--
Jon
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BtrFS Usage/Complaints

Jon Nelson-5
On Thu, Jul 4, 2013 at 1:40 PM, Jon Nelson <[hidden email]> wrote:
>
>> Lastly, I'd like to ask that you take the opportunity to test with the
>> latest openSUSE Factory kernel running 3.10-final. 12.3 used the 3.7
>> kernel since it was the most recent release going into the beta cycle
>> and we've seen a bunch of btrfs fixes since that release.
>
> I'll give it a try as soon as it's available. Are you talking about
> the OBS project with title:
> "Kernel builds for branch stable (standard)" ?


Unfortunately (and this appears to have been reported elsewhere) 3.10
runs super hot for me on my Thinkpad T520.
Over 152F (normally it's 115F to 120F).



--
Jon
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BtrFS Usage/Complaints

Lew Wolfgang
In reply to this post by Jeff Mahoney
On 07/04/2013 08:54 AM, Jeff Mahoney wrote:
> I look forward to your feedback and the opportunity to improve btrfs for
> the 13.1 release and future releases.

Hi Jeff,

I experimented with btrfs just a few days ago and bumped
into a fatal problem.  Our application calls for writing
data as fast as possible to a hardware RAID-6 array.  The
data comes from as many as four Gig-E Ethernet tuned to
run at 950-Mb/sec steady-state.  So we need to record as
much as 475-MBytes/sec.  We've been doing this for years
with two Ethernet channels and we wanted to test to see
if the boxes would handle four.  We've been using XFS
without issue in the past.

So I built a RAID-6 array with eleven 2-TB Seagate Constellation
disks.  An additional disk is used as a hot-spare.  The array
grosses out at 17162760M with XFS as reported by "df -BM".
I then write 4000 4-GB files and use the wall-clock time
to calculate MB/sec.  The files are all identical, but contain
random binary data.

This all on a fresh install of openSuSE 12.3 with a subsequent
zypper -dup.  The hardware has two Xeon E5-2643 cpus running at
3.3GHz.  The box has 64-GB of ram with a LSI MegaRAID SAS 9271-8i
raid controller in a 24-bay chassis.  System disks are two Intel SSD's
configured as RAID-1.

The XFS array writes at 973-MB/sec and df -h shows:

   /dev/sdc1        17T   16T  761G  96% /export/data1

I tried reiserfs but it wouldn't build on the 17-TB array,
it supports only up to 16-TB I guess.

I then formatted a second identical array using yast2 with
btrfs.  But the write test failed about 55% of the way
through.  The system was still operable, but the writing
just stopped.  /var/log/messages had kernel problems with
btrfs and generated a trace.  I've included a sample below.
The hostname is "beef".  A second attempt stopped in
approximately the same place.

The write rates for btrfs seemed to be about twice as fast
as XFS, but was irregular.

I then repeated the process with ext4 and got 992-MB/sec,
but the filesystem filled up.  It looks like ext4 uses
about 800-GB more space for its overhead than XFS, for
a 17-TB array.  So for now, we'll stick with XFS since
we have enough bandwidth, but can use the additional
storage space.

Regards,
Lew

beef's /var/log/message output showing btrfs crash:

2013-07-02T16:28:20.899462+00:00 beef kernel: [75249.184041] use_block_rsv: 10505
callbacks suppressed
2013-07-02T16:28:20.899478+00:00 beef kernel: [75249.184056] btrfs: block rsv
returned -28
2013-07-02T16:28:20.899479+00:00 beef kernel: [75249.184057] ------------[ cut here
]------------
2013-07-02T16:28:20.899480+00:00 beef kernel: [75249.184073] WARNING: at
/home/abuild/rpmbuild/BUILD/kernel-desktop-3.7.10/linux-3.7/fs/btrfs/extent-tree.c:6297
btrfs_alloc_free_block+0x3b1/0x3c0 [btrfs]()
2013-07-02T16:28:20.899481+00:00 beef kernel: [75249.184074] Hardware name:
X9DRH-7TF/7F/iTF/iF
2013-07-02T16:28:20.899482+00:00 beef kernel: [75249.184075] Modules linked in:
mpt2sas raid_class mptctl mptbase btrfs zlib_deflate libcrc32c st sr_mod cdrom dm_mod
minix hfs vfat fat af_packet xfs mperf joydev coretemp kvm_intel kvm crc32c_intel
ghash_clmulni_intel aesni_intel ablk_helper cryptd lrw aes_x86_64 xts gf128mul
iTCO_wdt gpio_ich iTCO_vendor_support microcode pcspkr sb_edac edac_core i2c_i801
lpc_ich mfd_core mei igb ses enclosure ptp pps_core sg ioatdma dca container button
autofs4 reiserfs mgag200 ttm drm_kms_helper drm isci i2c_algo_bit sysimgblt
sysfillrect syscopyarea libsas scsi_transport_sas processor thermal_sys scsi_dh_alua
scsi_dh_hp_sw scsi_dh_emc scsi_dh_rdac scsi_dh megaraid_sas
2013-07-02T16:28:20.899483+00:00 beef kernel: [75249.184113] Pid: 28610, comm: rm
Tainted: G        W    3.7.10-1.16-desktop #1
2013-07-02T16:28:20.899483+00:00 beef kernel: [75249.184114] Call Trace:
2013-07-02T16:28:20.899484+00:00 beef kernel: [75249.184124] [<ffffffff81004818>]
dump_trace+0x88/0x300
2013-07-02T16:28:20.899485+00:00 beef kernel: [75249.184129] [<ffffffff8158af33>]
dump_stack+0x69/0x6f
2013-07-02T16:28:20.899486+00:00 beef kernel: [75249.184134] [<ffffffff81045249>]
warn_slowpath_common+0x79/0xc0
2013-07-02T16:28:20.899486+00:00 beef kernel: [75249.184141] [<ffffffffa05d13b1>]
btrfs_alloc_free_block+0x3b1/0x3c0 [btrfs]
2013-07-02T16:28:20.899487+00:00 beef kernel: [75249.184172] [<ffffffffa05bd3d7>]
__btrfs_cow_block+0x137/0x570 [btrfs]
2013-07-02T16:28:20.899488+00:00 beef kernel: [75249.184183] [<ffffffffa05bd98f>]
btrfs_cow_block+0xff/0x250 [btrfs]
2013-07-02T16:28:20.899488+00:00 beef kernel: [75249.184195] [<ffffffffa05c1f99>]
btrfs_search_slot+0x429/0x990 [btrfs]
2013-07-02T16:28:20.899489+00:00 beef kernel: [75249.184209] [<ffffffffa05d7ab5>]
btrfs_del_csums+0x125/0x300 [btrfs]
2013-07-02T16:28:20.899490+00:00 beef kernel: [75249.184230] [<ffffffffa05caedf>]
__btrfs_free_extent+0x60f/0x890 [btrfs]
2013-07-02T16:28:20.899490+00:00 beef kernel: [75249.184246] [<ffffffffa05cf852>]
run_clustered_refs+0x322/0xac0 [btrfs]
2013-07-02T16:28:20.899491+00:00 beef kernel: [75249.184264] [<ffffffffa05d2d5a>]
btrfs_run_delayed_refs+0xca/0x320 [btrfs]
2013-07-02T16:28:20.899491+00:00 beef kernel: [75249.184293] [<ffffffffa05e3343>]
__btrfs_end_transaction+0x123/0x420 [btrfs]
2013-07-02T16:28:20.899491+00:00 beef kernel: [75249.184321] [<ffffffffa05edd14>]
btrfs_evict_inode+0x334/0x380 [btrfs]
2013-07-02T16:28:20.899492+00:00 beef kernel: [75249.184361] [<ffffffff811887c7>]
evict+0xa7/0x1a0
2013-07-02T16:28:20.899492+00:00 beef kernel: [75249.184373] [<ffffffff8117e37d>]
do_unlinkat+0x12d/0x1d0
2013-07-02T16:28:20.899498+00:00 beef kernel: [75249.184385] [<ffffffff8159eaad>]
system_call_fastpath+0x1a/0x1f
2013-07-02T16:28:20.899499+00:00 beef kernel: [75249.184390] [<00007f21a89400ed>]
0x7f21a89400ec
2013-07-02T16:28:20.899500+00:00 beef kernel: [75249.184391] ---[ end trace
c2679e952b898a26 ]---
2013-07-02T16:28:20.900367+00:00 beef kernel: [75249.184578] btrfs: block rsv
returned -28
2013-07-02T16:28:20.900373+00:00 beef kernel: [75249.184579] ------------[ cut here
]------------


--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BtrFS Usage/Complaints

Cristian Rodríguez-2
El 04/07/13 15:22, Lew Wolfgang escribió:

> beef's /var/log/message output showing btrfs crash:

that's a bug, please file it in bugzilla so it does not get lost in the
sea of -factory mail.

--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BtrFS Usage/Complaints

Hans Witvliet-2
In reply to this post by Jeff Mahoney
-----Original Message-----
From: Jeff Mahoney <[hidden email]>
To: [hidden email]
Cc: David Sterba <[hidden email]>, Vojtech Pavlik <[hidden email]>,
Mark Fasheh <[hidden email]>, Matthias Eckermann <[hidden email]>
Subject: [opensuse-factory] BtrFS Usage/Complaints
Date: Thu, 04 Jul 2013 11:54:05 -0400

Hi all -

Over the past year, btrfs has received substantial attention in the
areas of both stability and performance. I've been running it,
personally, on a 4 TB data volume as well as my root partitions for my
openSUSE 12.2/12.3 systems. I've been able to do stress tests like a
long fsmark run simultaneously with creating and removing snapshots
(with syncs to force them to disk) and haven't run into any issues.
David Sterba, who's been leading the charge at SUSE for btrfs stability
has likewise regularly run it through an array of torture tests and has
found that problems are occurring a whole lot less frequently than they
have in the past. The upstream btrfs community in which we participate
has also started focusing less on feature development and more on
stability and performance.

But these are only anecdotal data points from two file system guys and
if my experiences working for a major operating systems vendor have
taught me anything, it's that our users can be a lot more creative at
finding ways to break things than we are. ;)

So, I'd like to hear your stories. What's worked for you? What hasn't
worked? What would you consider the pain points with using btrfs?

-----Original Message-----

Hi Jeff,

Been using btrfs for quite some time now (release 12.2) for my data storage.
I've noticed that (for a given amount of storage) it has less overhead compared with others, like ext4.

Also i noticed, that if you fills it up completely (at least as root), you get a trace back in syslog.
Seems informational, cause it just keeps working.

What surprised me initially, that when you use an logical volume with ext4, you enlarge the LV and do
resizing the filesystem, you specify again the LV. When doing this for a LV filled with btrfs, you
start resizing the LV, but for btrfs, you specify the mountpoint.

Finally, it scared the hell out of me seeing that for each btrfs-mountpoint, several processes are created:
ps wwwaux |grep -v grep|grep btrfs |wc -l
554

for:
cat /proc/mounts |grep btrfs -c
36

So, as far as i can see, btrfs works right out of the box.

Hans





--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BtrFS Usage/Complaints

Catalin Iacob
In reply to this post by Jeff Mahoney
On Thu, Jul 4, 2013 at 5:54 PM, Jeff Mahoney <[hidden email]> wrote:
> So, I'd like to hear your stories. What's worked for you? What hasn't
> worked? What would you consider the pain points with using btrfs?

I ran it exclusively for about 6 months with the openSUSE 12.1 kernel
(3.1) and then a self compiled one going through 3.2 rcs, 3.3 rcs and
3.4 rcs.

I had performance issues with it, reported them upstream but didn't
get far (after a reply from Josef Bacik the thread died). You can see
the upstream thread which includes lots of details at
http://thread.gmane.org/gmane.comp.file-systems.btrfs/17638

Note that the thread above talks about slow mounts but it just
generally got slow as time passed. I reported the slow mount
particularly because it seemed like a discrete operation that was easy
to time and diagnose separately from everything else, the plan was to
get that fixed and then look into the runtime performance issues.

At the time I had the feeling that what was different between me and
other people that were running btrfs day to day was that I was using
snapper (therefore taking lots of snapshots automatically, hourly I
think) and lzo compression (which was quite new at the time).

The runtime performance issues seemed like huge lantency for IO
operations which ended up being unbearable. I remember a git rebase -i
(granted on the big Libreoffice repo) taking something on the order of
a minute just to come up with the list of my patches that needed to be
rebased on top of origin/master, while the disk was spinning all the
time. In general, there were lots of times when windows seemed
unresponsive, Firefox would freeze etc. all while the disk was
continuously spinning. Some seconds later (not uncommonly 10, 20, 30
seconds later) everything would respond again after disk activity
stopped.

I'm quite sure that it got progressively worse (which made me suspect
snapper), it definitely wasn't unbearable the first day after
installation, and it definitely got unbearable the day I decided to
format and switch to ext4. I say I'm sure it got progressively slower
because after switching to ext4 it literally seemed like I got a new
computer which was orders of magnitude faster. I'm sure I bared the
slowness only because the water the frog was in got gradually warmer
and so it took a while to realize how bad it had gotten.

So I'm a bit worried that after months of use performance degrades
significantly. Do you know of somebody that ran with lots of snapshots
and lzo for some months with no issues?

Hope this helps
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BtrFS Usage/Complaints

Frederic Crozat-4
In reply to this post by Graham Anderson-3
Le jeudi 04 juillet 2013 à 18:26 +0100, Graham Anderson a écrit :

> On Thursday 04 Jul 2013 11:54:05 Jeff Mahoney wrote:
> > So, I'd like to hear your stories. What's worked for you? What hasn't
> > worked? What would you consider the pain points with using btrfs?
>
> When I tested a variety of KVM image formats and settings to see what worked
> well with regards to disk read/write speed; I found that any disk image stored
> on a btrfs volume made writing to the the VM filesystem (ext4) too slow to be
> usable for any kind of real work load.
>
> I didn't test using btrfs volumes inside the VM's themselves.

I'm running btrfs on such setup and the way to get decent performance
(either when using only btrfs on host on in both host and guest) is to
disable CoW on the VM images, using "chattr -C". I'm wondering if we
shouldn't do that by default in qemu-img and libvirt..

--
Frederic Crozat <[hidden email]>
SUSE

--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BtrFS Usage/Complaints

Erwin Van de Velde
In reply to this post by Jeff Mahoney
Hi all,

My laptop is running on a btrfs root filesystem every since oS 12.2. I have had
two issues in the beginning, but now it is running perfectly for more than 6
months already. I use the latest kernel though (i.e. now 3.10). Since oS 12.3
I do not use a separate boot partition anymore, without any issues.

Kind regards,
Erwin

--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BtrFS Usage/Complaints

Ludwig Nussel
In reply to this post by Catalin Iacob
Catalin Iacob wrote:
> I'm quite sure that it got progressively worse (which made me suspect
> snapper), it definitely wasn't unbearable the first day after
> installation, and it definitely got unbearable the day I decided to
> format and switch to ext4. I say I'm sure it got progressively slower
> because after switching to ext4 it literally seemed like I got a new
> computer which was orders of magnitude faster. I'm sure I bared the
> slowness only because the water the frog was in got gradually warmer
> and so it took a while to realize how bad it had gotten.

I got exactly the same experience, both on a Laptop with encrypted
root volume as well as in a VM I only used for testing. Those delays
and slowness made me really angry about how bad Linux on the Desktop
has gotten until I realized it was just btrfs. I'll stay with ext4 now.

cu
Ludwig

--
  (o_   Ludwig Nussel
  //\
  V_/_  http://www.suse.de/
SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg)
--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BtrFS Usage/Complaints

Matthias G. Eckermann-3
In reply to this post by Erwin Van de Velde
Hello Erwin and all,

On 2013-07-05 T 10:14 +0200 Erwin Van de Velde wrote:
 
> My laptop is running on a btrfs root filesystem every
> since oS 12.2. I have had two issues in the
> beginning, but now it is running perfectly for more
> than 6 months already. I use the latest kernel though
> (i.e. now 3.10). Since oS 12.3 I do not use a
> separate boot partition anymore, without any issues.

just for my curiousity, two questions:

1. How big is your "/" partition?

2. Are you using Snapper, and if yes,
   do you use the default settings in
   "/etc/snapper/configs/root" or did you
   configure a more aggressive deletion
   of snapshots?
   
Thanks in advance!

so long -
        MgE

--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BtrFS Usage/Complaints

Agustin Benito Bethencourt
In reply to this post by Jeff Mahoney

Hi,

 

in my company laptop (Lenovo X220) I installed openSUSE 12.3 RC2 with btrfs then I updated to 12.3. I had an encrypted partition.

 

The performance was very good at the beginning. But it became worse and worse over time due to my hardisk being busy for no reason. At some point I began to experience time slots up to one minute in which I my laptop got frozen and came back to life again like nothing happened. I thought it was due to Akonadi. So I deactivated it but problems remained.

 

Ludwig Nussel checked it and determined that the source of the problem most likely was btrFS.

 

I reinstalled my machine some days ago with ext4 and full hard disk encrypted and so far the laptop goes fine. Nothing different from BtrFS behavior at this point in time for me though.

 

Please remember that I am not an engineer and my knowledge about the system is very limited.

 

On Thursday 04 July 2013 11:54:05 Jeff Mahoney wrote:

> Hi all -

>

> Over the past year, btrfs has received substantial attention in the

> areas of both stability and performance. I've been running it,

> personally, on a 4 TB data volume as well as my root partitions for my

> openSUSE 12.2/12.3 systems. I've been able to do stress tests like a

> long fsmark run simultaneously with creating and removing snapshots

> (with syncs to force them to disk) and haven't run into any issues.

> David Sterba, who's been leading the charge at SUSE for btrfs stability

> has likewise regularly run it through an array of torture tests and has

> found that problems are occurring a whole lot less frequently than they

> have in the past. The upstream btrfs community in which we participate

> has also started focusing less on feature development and more on

> stability and performance.

>

> But these are only anecdotal data points from two file system guys and

> if my experiences working for a major operating systems vendor have

> taught me anything, it's that our users can be a lot more creative at

> finding ways to break things than we are. ;)

>

> So, I'd like to hear your stories. What's worked for you? What hasn't

> worked? What would you consider the pain points with using btrfs?

>

> Lastly, I'd like to ask that you take the opportunity to test with the

> latest openSUSE Factory kernel running 3.10-final. 12.3 used the 3.7

> kernel since it was the most recent release going into the beta cycle

> and we've seen a bunch of btrfs fixes since that release.

>

> I look forward to your feedback and the opportunity to improve btrfs for

> the 13.1 release and future releases.

>

> Thanks!

>

> -Jeff

--

Agustin Benito Bethencourt

openSUSE Team Lead at SUSE

[hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BtrFS Usage/Complaints

Matthias G. Eckermann-3
In reply to this post by Graham Anderson-3
Hello Graham and all,

On 2013-07-04 T 18:03 +0100 Graham Anderson wrote:

> On Thursday 04 Jul 2013 11:54:05 Jeff Mahoney wrote:
>
> > So, I'd like to hear your stories. What's worked for
> > you? What hasn't worked? What would you consider the
> > pain points with using btrfs?
>
> On a several desktops and two small hosts and several
> VMs with limited size (20GB) root volumes, I had to
> disable/remove snapper and manually delete all the
> snapper created sub volumes. Without warning, the root
> volumes became un- writeable with write operations from
> various services and applications reporting no available
> space.
>
> I understand why df/du and monitoring tools are unable
> to correctly report free space in the same way that we
> are used to with more traditional non COW filesystems;
> but unless we educate users/admins about "btrfs
> filesystem df" or improve the various traditional
> reporting tools I bet the problem is going to bite more
> people.
>
> I didn't investigate the problem further to determine
> whether it was a lack of space for meta
> information/extents or whether the sheer number of
> subvolumes (even with COW) was taking up the space.
>
> The problem generally manifested itself after a medium
> period of about 3-4 months use, but on one desktop and
> one VM which were heavily used for testing packages, the
> problem appeared within a few weeks. Not good at all...

Admittedly, this is the concern I hear most often, and it
is (at least to a certain degree) based on too "snapshot
friendly" settings in the Snapper configuration for "/"
(root). Or to put it differently: Snapper should delete
older snapshots much more frequently and aggressivly.

I would propose a Snapper update to enforce this also on
12.2 and 12.3.  Something like this (pseudo code):

if ( Snapper-config-for-"root" has default values ) {
        Change to more aggressive values
} else {
        Do not touch the configuration, but issue a warning
        that the administrator should change this manually
}

What do you think?

[...]
 

> The second issue I ran into was a while ago and I
> apologise for no bug report on this one (I had no clue
> if btrfs was even officially supported in yast at the
> time). I can't remember the specifics but I was manually
> creating a btrfs filesystem with either multiple
> physical disks or md devices. The system partitioner
> (yast) was unable to recognise the filesystem; but
> what's worse is that I was able to add a btrfs
> filesystem via yast to the same physical or md devices
> that I had already manually created an fs on.

That sounds weird. If you run into it again, please create
a bugreport.
 
> In general, btrfs support in yast appears fairly limited
> and I would imagine I'd get really pissed off in future
> if the only supported brtfs use was via yast and that
> didn't allow for what I imagine will be fairly popular
> btrfs usage (multiple physical devices for large storage
> system). So I guess what I'm saying is I'd love to see
> some more love for btrfs in YAST with better support for
> btrfs options ;)

Everything needs a start -- so did btrfs support in the
YaST partitioner.

I see a request for RAID 1 support in openFATE:
        https://features.opensuse.org/313130
and one about improved subvolume handling:
        https://features.opensuse.org/314606

Going forward: feel free to add your requests to the
openFATE database, ...

So long -
        MgE

--
To unsubscribe, e-mail: [hidden email]
To contact the owner, e-mail: [hidden email]

123
Loading...