bcachefs

Fastest ordered key value store?

Based on all the numbers I've seen, it looks like bcachefs's b-tree might actually be the fastest ordered key value store around (there are faster persistent hash tables).

If anyone knows of anything that might be faster, I'd love to hear it - and I might rig up some head to head benchm...

More bcachefs btree benchmarks

rand_mixed is 3/4 lookups, 1/4 update

Lookups, both sequential and random, are beautifully fast.

Random updates ought to scale better than that, but I haven't profiled it yet so I'm not sure what's going on.

Could probably do better on sequential inserts/deletes too, but I haven't even...

Doing some pure btree performance tests

I should have done this ages ago...

bcachefs: btree_perf_test() doing 10.0M rand_insert:

bcachefs: btree_perf_test() done in 27 sec, 2587 nsec per iter, 377k per sec

bcachefs: btree_perf_test() doing 10.0M rand_lookup:

bcachefs: btree_perf_test() done in 10 sec, 1001 n...

Filesystem resize is implemented

You can now expand a filesystem on a device - shrink isn't implemented just yet. The command is bcachefs device resize, and it takes the same arguments as resize2fs:

bcachefs device resize /dev/sdb 10G

If you don't specify a size, it uses the current size of the device.

Online and offl...

Update and new bcachefs fs usage command

Replication tests are finally all passing! This means that device removal and write error handling (for replicated writes) should finally be fully working.

Those two codepaths have in common that they need to modify pointers to existing btree nodes - removing the pointer to the device that either ...

v4.13 rebase is up

Per inode checksumming, compression options

Just pushed out a patch to add per-inode options for some options that could previously only be set globally. Currently this is just checksum type and compression type, but more will be added in the future. The options are exposed as xattrs, and if you set them on a directory they'll be inherited on...

What's cooking

I've been back to work for several months but procrastinating and neglecting the updates...

The biggest thing I've been spending my time on lately has been improving the test infrastructure and test suite and chasing bugs. It seems I was putting this off for far too long, relying just on xfstests ...

Status update

Took this earlier today in Nashville :)

On the bcachefs front - might be announcing a corporate sponsor in the next few days! Stay tuned.

Status update

Status update - moving, portability/FUSE

If you've noticed things have been quieter lately, you haven't been imagining things - I've been busy with getting ready for a rather big move, and in the process I'm taking an extended road trip. I'm pretty happy about it - I've been feeling stuck in a rut and it's been hard to make progress writin...

Replication update

As I think I mentioned awhile ago, for replication the last big item left was IO error handling - that is, handling IO errors without just going read only when we've got another replica to read from (for reads) or when only some of the replicas for a replicated write failed.

The really tricky...

Faster fsck and mount times

Recently pushed a patch to add prefetching of btree nodes. It's a rather minor change compared to the stuff I'm still working on for replication, but it does improve both mount and fsck times by around 2x - not too shabby for a relatively simple change.

On larger filesystems, bcachefs's mount...

Status update - debugging and replication

Debugging, debugging, more debugging...

If you've been wondering at the slow progress, that's where all my time's been going. The unfortunate reality about creating a filesystem is that a filesystem, much moreso than most software, isn't all that useful if it's only, say, 90% debugged - you d...

New website, git repositories

http://bcachefs.org/

Don't have any _new_ content there yet, it's just all the existing stuff in one place. Would love to have people help out on the website.

Also, bcachefs has its own git repositories now - also linked to b...

Lots of new changes/features:

It's been far too long since the last announcement - lots of stuff has been

happening. The biggest milestone has been all the breaking on disk format

changes finally landing, but there's been lots of other stuff going on, too.

On the subject of the breaking on disk format ch...

Performance improvements in bcachefs-testing

First off, some background on where we're at currently, regarding metadata IO:

 - A userspace process will never block on IO - i.e., wait for a journal write or a btree node write - unnecessarily. Never ever.  The only reason your userspace proccess will end up blocked waiting for a meta...

Testing needed

Lately, the big bottleneck is getting to be testing - I really need more people willing to try out the latest code and make sure it isn't going to eat anyone's data before I push it out for general consumption. I do a lot of testing myself already - honestly, that's where most of my time goes - but ...

Upcoming performance improvements

These patches haven't landed yet, and the numbers should be higher when I'm done - but the fsmark numbers are now looking really nice. Delete performance is massively improved, too.

Time to completion for fs_mark -v -n 200000 -s 4096 -k -S 1 -D 1000 -N 1000 -t 10:

bcachefs: ...

Tiering should now be working - testing requested

Tiering should finally be working with the last big batch of fixes I pushed.

Chris Halse Rogers (RAOF in the #bcache IRC channel) has been testing it. He has been seeing an intermittent deadlock while copying large amounts of data, which may or may not be tiering related: if anyone else hits it, I...

All about tiering

First off, a word about definitions. In bcachefs, tiering is caching by another name: storage devices can be assigned to different tiers, and we can use a faster tier to cache a slower tier.

In some other storage systems, tiering means a setup where data can be dynamically moved between different ...

Encryption - motivation

Just saw this really excellent article about disk encryption - this explains better than I could the issues with encryption at the block layer:

http://sockpuppet.org/blog/2014/04/30/you-dont-want-xts/<...

Updates

 - Encryption's mostly done, got some useful feedback from the design doc.

 - Starting to work on multiple devices and replication again. Found some "there's no way this could have possibly worked" bugs with tiering - evidently I've neglected all the multiple device stuff for too long. 2016-09-05 02:40:02 +0000 UTC View Post

Encryption design doc is finished

https://bcache.evilpiepirate.org/Encryption/

Encryption

Been studying random papers/RFCs/Dan Bernstein's code and figuring out the plan for adding encryption to bcachefs... doing crypto right is hard. In storage land, I'm not sure anyone really gets it right - if you're doing block storage (e.g. dm-crypt), or if you're adding encryption to an existing fi...

Compression + copygc

I'm kicking myself for not noticing this sooner (most likely I saw it months ago and then forgot about it because I'm terrible about taking notes...). Do not use bcachefs with compression enabled yet - if copygc ever has to run you'll very soon hit a BUG_ON().

The issue is that copygc will often h...

Transactions

This is my current project, so I thought I'd write something about it and how this area of bcachefs works.

So, for some background: every remotely modern filesystem has some sort of facility for what database people just call transactions - doing complex operations atomically, so that i...

Compression has landed

The last bit - disk space accounting - is finished, so it should actually be useful now.

Please test it out - I'd like to hear how well it's working for people.

Currently lz4 and gzip are supported, and lz4 is the recommended option. I'd like to add more compression algorithms in the...

July 2016 patron supported

Thank you so much for the support this month!

Compression

Finally figured out how to make compressed disk usage accounting work. It's a surprisingly thorny issue - I'll have to write more about it later.

The TL;DR is - disk usage is only allowed to increase when you're getting a disk reservation (which is also where you'd get -ENOSPC). We have to be ...