Welcome to Tactical Gamer

+ Reply to Thread
Page 1 of 2 12 LastLast
Results 1 to 15 of 17
Discussion: General Forums / Hardware & Software Discussion - Byte Level Replication under Linux - Does anyone here have any experience with any Linux based byte level replication packages? I'm
  1. #1


    Apophis's Avatar

    Join Date
    Oct 2001
    Location
    Northeast, USA
    Age
    39
    Posts
    9,963
    Blog Entries
    10

    Byte Level Replication under Linux

    Does anyone here have any experience with any Linux based byte level replication packages? I'm looking for something that is preferably free and open source. I need to take a VM server running on a FC6 host and do real-time byte replication of the .VMDK files over to a secondary FC6 VM host.

    I looked at a few commercial packages but would also like to explore open source alternatives. I'm about to start playing with Pratima but wasn't sure if any of you other Linux IT professionals have used other solutions.

    Diplomacy is the art of saying "good doggie" while looking for a bigger stick.

  2.  
  3. #2


    Join Date
    Jan 2006
    Location
    United states, TN
    Age
    39
    Posts
    3,072

    Re: Byte Level Replication under Linux

    You might can use "dd" under the command line. The command would be something like "dd if=/foo of=foogoeshere". Take a look at the man files to see if it will work for you.

    I am by no means an IT pro and definitely not a linux expert but I have used dd in my job to replicate images before and it worked quite well.
    Retired 6th DB

  4.  
  5. #3

    ednos's Avatar

    Join Date
    Dec 2006
    Location
    Massachusetts
    Posts
    4,242

    Re: Byte Level Replication under Linux

    I would strongly recommend `dd`, unless it's known not to work with this sort of file (i.e., backing up DRM-protected game discs). You can use `dd` in combination with `netcat`, or its secure brother, `snetcat` (or encrypt it yourself).

    I'm not an IT professional, so I'm not sure if this is useful advice. I don't know what a VMDK file is.
    The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt. ~ Bertrand Russell
    I have a tendency to key out three or four things and then let them battle for supremacy while I key, so there's a lot of backspacing as potential statements are slaughtered and eaten by the victors. ~ Magna Centipede
    Feel free to quote me. ~ Skylark

  6.  

     
  7. #4


    Apophis's Avatar

    Join Date
    Oct 2001
    Location
    Northeast, USA
    Age
    39
    Posts
    9,963
    Blog Entries
    10

    Re: Byte Level Replication under Linux

    DD isn't going to do real-time byte level replication. It's going to make an image of the disk that would need to be transferred. Doing this with multiple 100+ GB files is impractical. The only package I found that will do what I want is the NetVault: Replicator package from BakBone. Unfortunately we would be looking at around $12,000 in licensing.

    The only other solution I've found is using Unison over SSH. It can do block-level changes on files so it will limit the amount of data that will need to be transferred between the machine (and over "slow" links to a remote site). It's free and open source, so at least it will be within the corporate budget to roll out. It's not real-time, but depending on how much data gets changed, I might be able to run replicas every 15 minutes.

    Diplomacy is the art of saying "good doggie" while looking for a bigger stick.

  8.  
  9. #5


    Apophis's Avatar

    Join Date
    Oct 2001
    Location
    Northeast, USA
    Age
    39
    Posts
    9,963
    Blog Entries
    10

    Re: Byte Level Replication under Linux

    Let me clarify:

    In the storage/HA industry, byte level replication refers to only replicating bytes within a file or volume that have changed rather than the entire file or volume. It's considerably different than a byte level copy.

    Diplomacy is the art of saying "good doggie" while looking for a bigger stick.

  10.  
  11. #6

    ednos's Avatar

    Join Date
    Dec 2006
    Location
    Massachusetts
    Posts
    4,242

    Re: Byte Level Replication under Linux

    The impression I get from http://www.vmware.com/interfaces/vmdk.html is that these files are not updated in such a way that realtime transfer would be necessary. However, since apparently they are, is there a way to set up the environments themselves to monitor and report all changes to the environment (i.e., a kernel-space module)? (This would only be practical in a UNIX environment.)
    The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt. ~ Bertrand Russell
    I have a tendency to key out three or four things and then let them battle for supremacy while I key, so there's a lot of backspacing as potential statements are slaughtered and eaten by the victors. ~ Magna Centipede
    Feel free to quote me. ~ Skylark

  12.  

     
  13. #7


    Join Date
    Jan 2006
    Location
    United states, TN
    Age
    39
    Posts
    3,072

    Re: Byte Level Replication under Linux

    Quote Originally Posted by Apophis View Post
    DD isn't going to do real-time byte level replication. It's going to make an image of the disk that would need to be transferred. Doing this with multiple 100+ GB files is impractical. The only package I found that will do what I want is the NetVault: Replicator package from BakBone. Unfortunately we would be looking at around $12,000 in licensing.

    The only other solution I've found is using Unison over SSH. It can do block-level changes on files so it will limit the amount of data that will need to be transferred between the machine (and over "slow" links to a remote site). It's free and open source, so at least it will be within the corporate budget to roll out. It's not real-time, but depending on how much data gets changed, I might be able to run replicas every 15 minutes.
    Ah gotcha! Unison looks like a viable solution since it uses rsync. I use rsync at home to back up my wifes work computer to our server. The initial transfer is huge but after that is over with just the files that have changed get sync'ed.
    Retired 6th DB

  14.  
  15. #8

    ednos's Avatar

    Join Date
    Dec 2006
    Location
    Massachusetts
    Posts
    4,242

    Re: Byte Level Replication under Linux

    Unison might be troubled to change to meet your needs, as it appears to be halfway there. I wish I knew more about this stuff; I'd develop it myself.
    The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt. ~ Bertrand Russell
    I have a tendency to key out three or four things and then let them battle for supremacy while I key, so there's a lot of backspacing as potential statements are slaughtered and eaten by the victors. ~ Magna Centipede
    Feel free to quote me. ~ Skylark

  16.  
  17. #9


    Apophis's Avatar

    Join Date
    Oct 2001
    Location
    Northeast, USA
    Age
    39
    Posts
    9,963
    Blog Entries
    10

    Re: Byte Level Replication under Linux

    Quote Originally Posted by ednos View Post
    Unison might be troubled to change to meet your needs, as it appears to be halfway there. I wish I knew more about this stuff; I'd develop it myself.
    Well, Unison is based off rsync, which is something I've used rather extensively over the past many years. I used rsync to keep web content on two Solaris 2.6 x86 in sync for failover purposes. I was initially going to just use rsync, but Unison seems to be a better product.

    What do you mean about Unison being troubled to change to meet my needs? I'm not quite following you on that one.

    Diplomacy is the art of saying "good doggie" while looking for a bigger stick.

  18.  

     
  19. #10

    ednos's Avatar

    Join Date
    Dec 2006
    Location
    Massachusetts
    Posts
    4,242

    Re: Byte Level Replication under Linux

    I mean it should be only a moderately challenging project to modify it to monitor files in realtime and report changes, which is what I have interpreted as your requirement. Correct me if I am incorrect.
    The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt. ~ Bertrand Russell
    I have a tendency to key out three or four things and then let them battle for supremacy while I key, so there's a lot of backspacing as potential statements are slaughtered and eaten by the victors. ~ Magna Centipede
    Feel free to quote me. ~ Skylark

  20.  
  21. #11


    Apophis's Avatar

    Join Date
    Oct 2001
    Location
    Northeast, USA
    Age
    39
    Posts
    9,963
    Blog Entries
    10

    Re: Byte Level Replication under Linux

    Quote Originally Posted by ednos View Post
    I mean it should be only a moderately challenging project to modify it to monitor files in realtime and report changes, which is what I have interpreted as your requirement. Correct me if I am incorrect.
    Ahh.. Yes. I would like it to work in real-time, but that's not necessarily an absolute requirement of the project. If I proposed a 15 minute replication cycle, I'm sure they would be fine with it.

    I thought about using some code that monitors the inode tables (via loadable kernel module) to trigger the replication, but that's honestly more trouble than it's worth for me to do.

    Ahsay as a nice add-on to their OBS product that does real-time byte level replication. It's also pretty cheap, but unfortunately it only does replication between OBS servers and not between the file systems directly.

    At one point in time I looked at using Ahsay and OBS to provide off-site network backup services for TG supporting members, but it was cost-prohibitive. I could always support it as an add-on service for TG folks and do it at cost, but I'm not sure what the demand is for this type of service within TG.

    Diplomacy is the art of saying "good doggie" while looking for a bigger stick.

  22.  
  23. #12

    jepzilla's Avatar

    Join Date
    Jul 2005
    Location
    Canuckistan
    Age
    30
    Posts
    1,233

    Re: Byte Level Replication under Linux

    Quote Originally Posted by Apophis View Post
    I thought about using some code that monitors the inode tables (via loadable kernel module) to trigger the replication, but that's honestly more trouble than it's worth for me to do.
    And already been done. IIRC inotify has been in the kernel for a while now, and provides file-change monitoring services.

  24.  

     
  25. #13

    ednos's Avatar

    Join Date
    Dec 2006
    Location
    Massachusetts
    Posts
    4,242

    Re: Byte Level Replication under Linux

    Hmm, I've never seen it. Then again, I haven't used a kernel beyond version 2.6.14.
    The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt. ~ Bertrand Russell
    I have a tendency to key out three or four things and then let them battle for supremacy while I key, so there's a lot of backspacing as potential statements are slaughtered and eaten by the victors. ~ Magna Centipede
    Feel free to quote me. ~ Skylark

  26.  
  27. #14


    Apophis's Avatar

    Join Date
    Oct 2001
    Location
    Northeast, USA
    Age
    39
    Posts
    9,963
    Blog Entries
    10

    Re: Byte Level Replication under Linux

    Update:

    After quite a bit of research and product trials I came down to a few solutions.

    One of which was not actually replicating the data on our RHEL server but just doing normal byte level replication inside the Windows hosts that we run under VMware on the RHEL box. That proved to be costly as software like DoubleTake isn't cheap. We would also need a pair of licenses for every Windows host we wanted to replicate.

    I looked into another product called NetVault Replicator from BakTrack. This is truly a great product but it's quite pricey. It would do everything I needed but to the tune of $10,000 per pair of HA servers.

    In the end I ended up playing around with DRBD, Heartbeat and Open-iSCSI.

    Here's what I did:

    I set up two machines under VM as a test. Installed Fedora 7 on each of them and updated to the latest kernels. I then downloaded DRBD and got that configured on each of the machines. I gave DRBD a physical disk to manage and copied a bunch of files onto the disk on the Primary server. That data was replicated at the block level over to the Secondary server. I then promoted the Secondary server to Primary with DRBD and mounted the volume and all my files were there. Excellent.

    Once DRBD was fully tested I downloaded and installed Heartbeat V2. After a couple hours of reading and configuration, I had Heartbeat managing DRBD and the mount/dismount of the volumes. If I failed the primary server, Heartbeat would automatically promote the backup server to primary and mount the drive accordingly. Excellent. I now had automated failover between the two servers.

    I tested that for a bit to make sure it was working as planned and attempted various methods of "breaking" the replication to make sure it was stable and would recover from failure in a predictable manner.

    When all my testing was done. I went ahead and installed iSCSI on the box and set up fileio block volumes on the DRBD managed drive. I mounted the /dev/drbd0 drive under /mnt/array/ and created block volumes at /mnt/array/iscsi-target0.img, /mnt/array/iscsi-target1.img, and /mnt/array/iscsi-target2. Once the iSCSI targets were created, I used the wonderful (heh, right) iSCSI Initiator 2.04 on my XP machine and mounted the targets. Once mounted, I was able to copy a bunch of files from a local drive to one of the iSCSI drives just as you would to a local drive.

    My final failover testing consisted of taking the primary offline and verifying that Heartbeat properly mounted the /dev/drbd0 drive on the secondary and then fired up the iSCSI daemon to export the targets. All worked well. Bringing the primary back on-line automatically triggered a reverse replication from the secondary back to the primary before stopping iSCSI and unmounting the drives on the secondary and then mounting the drives and starting iSCSI on the newly recovered primary.

    The only issue I have now is getting the Microsoft iSCSI Initiator to look at at the virtual IP address that Heartbeat tosses back and forth between the servers rather than the physical IP addresses of the two boxes. It appears to just be a Microsoft iSCSI implementation issue, but I need to do a bit more testing to confirm that.

    If anyone is in a position where they need to build extremely low-cost high availability file servers, this is a great way to go. I'd gladly share my config files and any tweaks I had to perform to get this running.

    Diplomacy is the art of saying "good doggie" while looking for a bigger stick.

  28.  
  29. #15

    ednos's Avatar

    Join Date
    Dec 2006
    Location
    Massachusetts
    Posts
    4,242

    Re: Byte Level Replication under Linux

    Sweet! I was just searching for this thread about a week ago, since we might need a similar solution at work. I also want to play with this stuff on my own computers because it's so damn *cool*.
    The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt. ~ Bertrand Russell
    I have a tendency to key out three or four things and then let them battle for supremacy while I key, so there's a lot of backspacing as potential statements are slaughtered and eaten by the victors. ~ Magna Centipede
    Feel free to quote me. ~ Skylark

  30.  

     

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts


  
 

Back to top