✎ soc:2010:peper:journal:week10

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.

====== Piotr Jaroszyński: Usermode debugging under Linux ======
===== Week 10 [ Jul 26 - Aug 1 2010 ] =====

==== drivers in userspace ====
There is one thing that keeps bugging me in the [[http://git.etherboot.org/?p=people/peper/gpxe.git;a=shortlog;h=refs/heads/drivers|drivers branch]] - UIO-DMA and malloc.
UIO-DMA does the DMA mappings on a per device basis (as does kernel). I am
not sure whether the mappings are really per device or global, but the
API suggests the former or at least that it might change in the future
(if it didn't with IOMMUs already). Anyway, because of that UIO-DMA
needs a device to be initialized and that's why I added a very simple
[[http://github.com/peper/uio-dma/blob/gpxe/kmod/uio-dma-gpxe.c#L25|uio-dma-gpxe module which does just that]]. This isn't really an
issue on its own (it might be even a plus as it forces to unbind any
other drivers first), but it complicates the initialization logic in
gPXE. Currently all the userspace setup of pci devices is done by the
linux lpci driver. And that's way too late to setup malloc. To work
around that I have added an option to use the last device setup with
UIO-DMA for the mappings and introduced a UIO-DMA malloc_backend - [[http://git.etherboot.org/?p=people/peper/gpxe.git;a=commitdiff;h=14009a8a96431cecd089c70d6ac44bfe75244497|here]] and [[ http://git.etherboot.org/?p=people/peper/gpxe.git;a=commitdiff;h=941ffdafdac66bcbca04fbac1668a4b944c7c18a|here]].
The reason I use the tables API (and not single API) for the backends
is that it needs to be able to fallback so that using the tap driver
is still possible without the UIO-DMA modules being loaded and initialized).

I don't really like the solution and I am starting to think that doing
a completely separate malloc with support for switching the memory
pool used for DMA allocations (again, so that tap works) is the way to
go. The reason I was reluctant to do so is that making the current
allocator valgrindable took me a fair amount of time. Or I could look
into making current malloc support that, but that will surely grow the
code size.

Or I could make the malloc backend a single API (no need for fallback any more) and implement it in userspace like that:
  * in a __init_fn provide the malloc pool by mmap()ing a chunk of memory (like linux_umalloc does) - let's call it A
  * if an lpci device is going to be used:
     * allocate the same amount of memory as for A from UIO-DMA - let's call the new chunk B
     * copy A over to B
     * munmap() A
     * mremap() B to the now free location of A

I like the last option most cause it uses the current malloc() implementation, works for tap and doesn't feel so hackish (just sophisticated ;). I am going to implement that soonish unless other ideas arise.

Trace: • 2009 • week4 • 2006 • week4 • week8 • week5 • ubuntu_iscsi • week6 • week8 • week3 • debian_lenny_iscsi • iscsibootprepfc5 • win2k8_iscsi_install