[gPXE-devel] COW Overlay in WinVBlock -- willing to implement

[Miller, Shao]

Pre-kernel: Perhaps you could do gPXE -> read-only iSCSI SAN -> MBR ->
NTFS Partition -> NTLDR -> specially-crafted NTBOOTDD.SYS (for VHDs) ->
"Master" VHD

 

Post-kernel: MS iSCSI Initiator and Michael Brown's 'sanbootconf' driver
re-establish the SAN.  WinVBlock creates a new "child" VHD on a
partition on the local HDD, with the "parent" VHD still the same one on
the same iSCSI SAN.

 

Does this seem like it'd satisfy your goals?  Do you need anything to
persist beyond a reboot?  What is the OS you are hoping to boot on the
clients?

 

If gPXE development isn't really involved, perhaps I could request
private correspondence for any WinVBlock-related matters, if you please?
If tweaks need to be made to gPXE, then by all means, I would agree that
this forum would be appropriate. :-)

 

Thanks again for your interest!

 

- Shao Miller

________________________________

From: Kenton Varda [mailto:temporal at gmail.com] 
Sent: Wednesday, May 04, 2011 11:21
To: Miller, Shao
Cc: gpxe-devel at etherboot.org
Subject: Re: COW Overlay in WinVBlock -- willing to implement

 

I'm aware that I can set up COW on the server side, but I'd really
rather that writes didn't cross the network at all.  It's not uncommon
for guests to want to install additional games which can be as much as
10GB each, and my machines have 60GB local SSDs which I'd like to
utilize.

 

In fact, longer-term it would be nice to use those SSDs as read caches
in addition to write caches, maybe even sniffing blocks being sent to
other machines on the network and precaching them.  Typically, everyone
loads the same game at the same time, and games need to load a lot of
data, so it would be neat to avoid transferring that data over the
network 12 times.  That said, this doesn't appear to be a significant
performance problem in my existing setup, so it is not a priority.
(Actually, I have already written code that implements this idea as a
userspace daemon on top of NBD, though it seems to have some bugs
still.)

 

I'm also aware that commercial solutions exist, but I'd rather use an
open source solution that I can customize, e.g. with the features I
mentioned above.  Also, the commercial solutions I've found require
running a Windows server, which I'd like to avoid.  And anyway,
implementing this would be fun.  :)

 

-Kenton

 

On Wed, May 4, 2011 at 5:27 AM, Miller, Shao
<shao.miller at yrdsb.edu.on.ca> wrote:

Good day, Kenton.

 

I'm currently working on some serialization/deserialization code in
order to simplify developing support for Microsoft's [acquired] VHD disk
image format.  Once implemented, the "differencing" feature of VHD image
files will provide copy-on-write functionality.

 

For example, I'd like to be able to boot from a master VHD pre-kernel,
then have WinVBlock continue with a delta.

 

However!  Since you've e-mailed the gPXE developers' list, I suspect
that you are more interested in SAN CoW of some sort.  This is currently
possible using Linux' device-mapper, and serving the CoW devices.  Did
you see this article?:

 

  http://etherboot.org/wiki/appnotes/cow

 

CoW-capable SAN target software seems to be a popular wish.  A fellow
recently shared a fun video of Ardence:

 

  http://www.youtube.com/watch?v=moIuHqIc-PQ&feature=player_embedded

 

I'm not 100% sure how you might be thinking of using gPXE + WinVBlock,
or if you would even need WinVBlock.  Would you care to share a few more
details about the scenario you have envisioned?

 

Thank you for your offer to help with development.  When WinVBlock was
originally derived from V.'s WinAoE work, it was quite a learning curve,
having dwelled in user-land, myself.  But before suggesting anything to
jump into, perhaps a little more discussion regarding your scenario
would be better.

 

Thanks!

 

- Shao Miller

________________________________

From: Kenton Varda [mailto:temporal at gmail.com] 
Sent: Wednesday, May 04, 2011 01:35
To: Miller, Shao; gpxe-devel at etherboot.org
Subject: COW Overlay in WinVBlock -- willing to implement

 

Hi Shao and gpxe-devel,

 

In a post on this page you mentioned that you are planning to add COW
overlay support to WinVBlock:

  http://reboot.pro/8168/page__st__300

 

I have a need for this (in order to netboot multiple machines off a
common base image), and have not been able to find any existing
non-proprietary solution.  Can I help you implement it?  If so, can you
give me pointers on where to start and what to do?

 

I am fluent in userland C (including low-level Win32 and POSIX), but
have never ventured into kernel drivers before.  FWIW I'm a Google
employee, but this is for a personal project.  Namely, my house, which
is optimized for LAN parties.  I currently have 12 machines netbooting
Ubuntu over nbd and running games under WINE, but sadly there are a few
too many games that don't work.  :/

 

Thanks,

-Kenton

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://etherboot.org/pipermail/gpxe-devel/attachments/20110504/d36c9348/attachment.html>