[gPXE-devel] COW Overlay in WinVBlock -- willing to implement

[Kenton Varda]

I'm aware that I can set up COW on the server side, but I'd really rather
that writes didn't cross the network at all.  It's not uncommon for guests
to want to install additional games which can be as much as 10GB each, and
my machines have 60GB local SSDs which I'd like to utilize.

In fact, longer-term it would be nice to use those SSDs as read caches in
addition to write caches, maybe even sniffing blocks being sent to other
machines on the network and precaching them.  Typically, everyone loads the
same game at the same time, and games need to load a lot of data, so it
would be neat to avoid transferring that data over the network 12 times.
 That said, this doesn't appear to be a significant performance problem in
my existing setup, so it is not a priority.  (Actually, I have already
written code that implements this idea as a userspace daemon on top of NBD,
though it seems to have some bugs still.)

I'm also aware that commercial solutions exist, but I'd rather use an open
source solution that I can customize, e.g. with the features I mentioned
above.  Also, the commercial solutions I've found require running a Windows
server, which I'd like to avoid.  And anyway, implementing this would be
fun.  :)

-Kenton

On Wed, May 4, 2011 at 5:27 AM, Miller, Shao <shao.miller at yrdsb.edu.on.ca>wrote:

>  Good day, Kenton.
>
>
>
> I’m currently working on some serialization/deserialization code in order
> to simplify developing support for Microsoft’s [acquired] VHD disk image
> format.  Once implemented, the “differencing” feature of VHD image files
> will provide copy-on-write functionality.
>
>
>
> For example, I’d like to be able to boot from a master VHD pre-kernel, then
> have WinVBlock continue with a delta.
>
>
>
> However!  Since you’ve e-mailed the gPXE developers’ list, I suspect that
> you are more interested in SAN CoW of some sort.  This is currently possible
> using Linux’ device-mapper, and serving the CoW devices.  Did you see this
> article?:
>
>
>
>   http://etherboot.org/wiki/appnotes/cow
>
>
>
> CoW-capable SAN target software seems to be a popular wish.  A fellow
> recently shared a fun video of Ardence:
>
>
>
>   http://www.youtube.com/watch?v=moIuHqIc-PQ&feature=player_embedded
>
>
>
> I’m not 100% sure how you might be thinking of using gPXE + WinVBlock, or
> if you would even need WinVBlock.  Would you care to share a few more
> details about the scenario you have envisioned?
>
>
>
> Thank you for your offer to help with development.  When WinVBlock was
> originally derived from V.’s WinAoE work, it was quite a learning curve,
> having dwelled in user-land, myself.  But before suggesting anything to jump
> into, perhaps a little more discussion regarding your scenario would be
> better.
>
>
>
> Thanks!
>
>
>
> - Shao Miller
>  ------------------------------
>
> *From:* Kenton Varda [mailto:temporal at gmail.com]
> *Sent:* Wednesday, May 04, 2011 01:35
> *To:* Miller, Shao; gpxe-devel at etherboot.org
> *Subject:* COW Overlay in WinVBlock -- willing to implement
>
>
>
> Hi Shao and gpxe-devel,
>
>
>
> In a post on this page you mentioned that you are planning to add COW
> overlay support to WinVBlock:
>
>   http://reboot.pro/8168/page__st__300
>
>
>
> I have a need for this (in order to netboot multiple machines off a common
> base image), and have not been able to find any existing non-proprietary
> solution.  Can I help you implement it?  If so, can you give me pointers on
> where to start and what to do?
>
>
>
> I am fluent in userland C (including low-level Win32 and POSIX), but have
> never ventured into kernel drivers before.  FWIW I'm a Google employee, but
> this is for a personal project.  Namely, my house, which is optimized for
> LAN parties.  I currently have 12 machines netbooting Ubuntu over nbd and
> running games under WINE, but sadly there are a few too many games that
> don't work.  :/
>
>
>
> Thanks,
>
> -Kenton
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://etherboot.org/pipermail/gpxe-devel/attachments/20110504/70c274eb/attachment.html>