6.11 was an expensive forced on disk format upgrade, for the disk accounting rewrite: while bcachefs is still marked as experimental, I'm making on disk format changes that would not be feasible if we needed compatibility code so that new versions could work on old filesystems without upgrading t...
2024-11-29 22:26:43 +0000 UTC View PostFixes will have to come from my repository.
This came just as the CoC board seemed to be relenting, and we started to be having a public conversation. That's now been cut off, and I think after the private correspondence we had their action was dishonest.
Here's what I wrote to Michal...
2024-11-22 22:25:54 +0000 UTC View PostTLDR: the future of bcachefs in the kernel is uncertain, and lots of things aren't looking good.
Linus has said he isn't accepting my 6.13 pull request, per "an open issue with the CoC board", and at this point I have no idea what's going on with the CoC board. I, for my part, have felt for...
2024-11-21 02:14:29 +0000 UTC View PostSo, kernel programming is known for being a high stakes, high pressure, toxic environment to work in. There's a reason for that: the consequences for screwing up are really high, and you can make your co-workers and a lot of users really miserable if you screw things up.
I've seen plenty o...
Initial support for online fsck is merged - it's in my master branch, and will be in Linux 6.8. To use it, just run the normal fsck command; if the filesystem is mounted, it'll use the online codepath.
Not all fsck passes are safe to use while the filesystem is in use yet: online fsck only ...
2023-12-09 02:23:22 +0000 UTC View PostRecently, I added a new superblock section for tracking counts of every distinct filesystem error (i.e. fsck error) since filesystem creation, as well as the date of the most recent error.
The idea is that inconsistencies that fsck is able to repair often don't go reported - but they still ...
2023-11-25 20:09:50 +0000 UTC View PostPhoronix recently posted some bcachefs benchmarks, and their results looked a bit... off.
Here's what I just got, testing 4k random writes with a similar fio configuration. SSD is a Samsung 970 Evo plus. Default mkfs options for all three filesystems.
xfs: 1 job 456k iops, 8 jobs 548...
2023-11-04 15:26:48 +0000 UTC View PostSome recent features that have landed:
* Rebalance work btree: Rebalance no longer has to scan for extents that need the background_target or background_compression option applied. Instead, there's a new bitset btree, updated by the extent trigger, that rebalance uses to find extents ...
2023-10-29 17:03:43 +0000 UTC View PostSo, the upstreaming process has been rocky - but bcachefs is in linux-next! Fingers crossed for 6.7. Since being merged into the linux-next tree we've also been getting a lot more assorted bugfixes and bugreports, mostly from static analysis - every bit helps.
Some other recent work:
...
2023-09-22 15:13:01 +0000 UTC View PostLet's see, what is there to talk about since the last:
Mostly, it's been a whole lot of bug fixing and stabilizing, grinding away at test failures in the CI: https://evilpiepirate.org/~testdashboard/ci?branch=bcachefs
...
2023-03-29 14:57:29 +0000 UTC View PostThere's so much that goes into developing a real filesystem. Especially one that's intended to be good enough to replace our existing filesystems, codebases that have had decades of refinement by teams of engineers.
Some days it can feel a bit overwhelming.
A filesystem has to be fast...
2022-11-28 21:46:16 +0000 UTC View PostWe _finally_ have a server continuously running tests and outputting to a nice dashboard. I'm pretty excited - this is going to make my life a lot easier, and it's another thing people can look at to see the current status.
If anyone skilled in web development is interested in helping out, ...
2022-07-03 19:26:07 +0000 UTC View PostNew on disk format, required upgrade. Your existing filesystem will be automagically upgraded when you upgrade kernel & tools to the new version.
What this means: much more efficient copygc. In the future we'll be using it for other things too, like possibly accelerating rebalance. The ...
2022-06-15 19:05:46 +0000 UTC View PostReorganized my todo list last night, now automatically synced from my home directory:
https://evilpiepirate.org/~kent/.plan.txt
(Patreon's post submission tool doesn't pick up the link correctly, you'll have to edit it...
2022-04-25 17:14:50 +0000 UTC View PostI've been starting to work on support for zoned devices, laying out what needs to be done. This will get us native support for SMR (shingled magnetic recording) hard drives, and (even more exciting!) fancy new ZNS SSDs, which strip away most of the FTL: by folding that into the filesystem, we'll ...
2022-03-17 04:12:38 +0000 UTC View PostIt's a mandatory disk format upgrade; when switching to the new version on an
existing filesystem you'll see it initialize the freespace btree when you mount.
What's changed: we've got some new persistent data structures that replace code
that used to perio...
2022-03-14 00:49:18 +0000 UTC View PostBacon and eggs, tea, sitting down to work on finishing the BTREE_ITER_WITH_JOURNAL patch, but before I can get to the interesting and necessary algorithmic work I've got like half a dozen bug reports and problems to respond to. And people wonder why I haven't upstreamed yet..
- this i...
2021-12-28 20:58:14 +0000 UTC View PostBcachefs now has a user manual - check it out!
https://bcachefs.org/bcachefs-principles-of-operation.pdf
Still need to expand the sections on sysfs internals, and the on disk format.
2021-12-17 23:10:18 +0000 UTC View PostJust finished fixing a whole bunch of i_sectors accounting bugs - the count of how many sectors are in a given inode/file. It turns out our accounting on disk, in the btree was completely fine - fsck would've noticed, if it wasn't - but the in memory accounting is different due to dirty data in t...
2021-11-28 21:13:20 +0000 UTC View PostThey work similarly to btrfs snapshots. There's new bcachefs subcommands for creating subvolumes and snapshots.
We've got writable snapshots, snapshots of snapshots, and snapshot and subvolume deletion. Create as many snapshots as you want - you're only limited by the amount of disk space y...
2021-09-27 00:44:26 +0000 UTC View PostThere's a new on disk format change, and it's a required upgrade - you'll want to upgrade your kernel and tools at the same time to stay in sync.
t was The new update closes a hole in our ability to verify metadata that goes back to bcache: since btree nodes are log structured, historically...
2021-07-16 16:46:40 +0000 UTC View PostPlease report it _any_ time errors are reported on a filesystem, fsck errors or otherwise. There's a couple of nasty bugs I'm trying to track down right now (one of them appears to be btree writes getting lost, but so far the reports are only from filesystems that have replication enabled, and it...
2021-04-22 07:13:23 +0000 UTC View PostTurns out, there was a hiccup after all in the on disk format changes for snapshots: it turns out we were generating packed bkey formats where the snapshot field was too big, and this breaks the lookup code when we start using the snapshot field.
I just pushed out code to scan the btree for...
2021-03-25 00:13:29 +0000 UTC View PostThey're still in a very early state, and nothing but the bare minimum is complete - but they're up, for people to look at and try out. You'll need to use the snapshots branch from both the kernel and tools repositories.
Caveats:
- I haven't finished the compat code, you'll have ...
2021-03-24 00:54:30 +0000 UTC View PostSnapshots are really coming together - the whole thing is starting to look shockingly elegant and simple (you wouldn't be able to guess from the design doc how many years it took to get to that point...)
Comments welcome:
Recently, Dave Chinner's been helping out with some performance testing of bcachefs on his big dual socket machine, which has been great for finding scalability issues and he's also been able to provide really useful comparisons and microbenchmarks where bcachefs is still behind xfs, and I've bee...
2020-11-20 20:59:13 +0000 UTC View PostErasure coding is ready for wider testing!
I just finished reworking disk space accounting for parity blocks - they're broken out from user data now, so bcachefs fs usage will show you that overhead. And I've been fixing bugs; there shouldn't be any gaping holes, and I don't expect it to ca...
2020-10-24 00:58:18 +0000 UTC View PostFinally been making some good progress on erasure coding: currently debugging the new code to update existing stripes with new blocks, which is the main piece that was missing - this is needed to deal with internal fragmentation across erasure coded stripes and avoid running out of space.
...
