Talk:Memory and Filesystems

From gc-linux

Jump to: navigation, search

I not sure there's really any point in documenting the userspace nbd client in the Swap to Network section I would have thought the using a userspace driver when trying to evict pages in low memory conditions would break pretty badly.

By all means document using the userspace one but probably this needs to be somewhere else.

--Daniel Thompson 13:20, 8 Nov 2005 (PST)

You may find useful reading a bit more about nbd, and looking at the userland tools and kernel source code. The userland nbd client is just a tool which at the end ioctls into the kernel, and does nothing more (except catching signals). You could even swap the client. This is not different than what the in-kernel client does.

The problem with swapping to network is not related to userland or kernelland. The problem is that the network layer allocates memory to work. And if you get to the point that there's really no memory, bang (using netpoll could be a solution).

An additional problem with the userland nbd client is that it is user-killable. This makes it problematic:

  • on system shutdown (it can get killed before turning swap off unless init scripts are made aware of nbd swap)
  • if the oom killer is woken up and elects it
  • if a user decides to kill it

On the other hand, nbd-root users are already using the in-kernel nbd client, and can't use it for swap too. So if they want nbd based swap they _cannot_ follow your instructions. That's when the userland nbd client, with the necessary precautions, can get handy, and may deserve an explanation here.

I personally do not use nbd swap on my cube. aram swap is (always) enough for me.

--isobel 14:34, 8 Nov 2005 (PST)

Clearly this is something that someone needs to try. I do remember that when I first started using gc-linux I tried using the usermode client and failed. Sadly I can't remember what didn't work and it may have been my mistake. However given that the client process can be swapped out I thought it would be likely (or at least possible) that at some point the data the server required to successful swap in might itself be swapped out. Naturally we could put calls to mlock() into the server to prevent this but whether anyone will look at this soon I don't know.

--Daniel Thompson 11:06, 9 Nov 2005 (PST)

As commented before, it is the kernel side of nbd (CONFIG_BLK_DEV_NBD) who does the job on the client side, even if you use the regular userland client. And that kernel memory is not evicted.

Until I wrote the in-kernel nbd client, all that existed was the userland client. And people was already swapping to nbd, although not in an absolute 100% reliable manner beause of the reasons I outlined above.

Probably, during your tests, you met the oom killer, which btw used to kill in excess in not so distant kernel releases :)

--isobel 11:31, 9 Nov 2005 (PST)

Thanks for the clarification. When told to go read the code perhaps next time I'll go read the code.

--Daniel Thompson 13:56, 9 Nov 2005 (PST)

Personal tools